Archives
Contributors
30 Checkout Operations per Minute A job for Odoo or not ?
Hi Folks,
i have a project proposal on my desk that will ultimately end up in
approx. 30 checkout transactions per minute. Has anybody got a project
with that order volume done with a relatively recent version of Odoo
and provide a good reference. Please bear in mind that the project
ahead involves a similarly high volume of stock moves.
Any suggestion / comment like (but not limited to)
Odoo is the wrong technology for that, forget it, because....
over
You can do it but, a), b), c) ....
to
If you really want that beast flying involve us over here because we
are among the few ones that already did soemthing similar
would be appreciated.
From our load intensive projects at initOS i know that 10000 orders a
day can be handled but almost 50000 is a different story and thats why
i am asking the most experienced resource out there ;-)
Thanks in advance for your replay
Cheers Frederik
--
Dr.-Ing. Frederik Kramer
Geschäftsführer
initOS GmbH
An der Eisenbahn 1
21224 Rosengarten
Phone: +49 4105 56156-12
Fax: +49 4105 56156-10
Mobil: +49 179 3901819
Email: frederik.kramer@initos.com
Web: www.initos.com
Geschäftsführung:
Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke
Sitz der Gesellschaft: Rosengarten – Klecken
Amtsgericht Tostedt, HRB 205226
Steuer-Nr: 15/200/53247
USt-IdNr.: DE815580155
by Frederik Kramer - 10:16 - 28 Apr 2021
Follow-Ups
-
Re: 30 Checkout Operations per Minute A job for Odoo or not ?
El jue, 6 may 2021 a las 5:02, Frederik Kramer (<frederik.kramer@initos.com>) escribió:Hi Moises, Am Freitag, den 30.04.2021, 16:02 +0000 schrieb Moises Lopez: > Hi Frederik > > A customer requested to support 5k concurrent sale.order (1 or 2 > lines of products) in Odoo. > The process that we followed were: test, measure, fix, test... haha, yes thats what works well if you have a customer that well endorses this "agile" and "pivoting" style of work. Apart from all economic considerations this is probably also something that isolated leads to quite solid solutions
Well, we are talking about 5k concurrent VUs in this special case.The cost vs benefit demands this.> > The first time that we tested the checkout process supported 15 > concurrent virtual users (VUs) > The measure said that it had heavy custom functions in the same > transaction of checkout but it could be executed in background (cron, > queue) > and there were slow queries because of indexes issues or too many > queries executed because of python code did you use any specific profiling tools here? Or the ones that rcollet often proposes in his talks about the matter ?
Watch my talk in SFOI used the same tools> > Note: Be careful with the base_automation if you can migrate them to > code it is better. Thanks for the hint, i think we experienced that also quite some times in the past and learned that lesson already ;-) > > After fix them the next iteration supported 400 VUs > So good but so far from the focus. > > The next iteration we found that there were methods that could be > using LRU-cache (e.g. prices, website that don't change) > Now we needed to check where the cache was clearing too frequently > and fix it. How did you measure LRU cache hit rates vs. code parts in a good way. Any tooling suggestion here ?
For the case of the price in the website our reasoning was:- The value don't change in a long time for this customer (each 6 months)- It couldn't be a stored field in the database because it depends on:- date- quantity- currency rate (if your have multi-currency)- partner- Supplier price (if your price depends on it)- product category- Adding LRU-Cache with these keys will work fine and fasterEven the website the most of request came from public userI share you the output of the following command in production:- kill -SIGUSR1 WORKER-PIDIt shows the hit rates:- 19715 hit 91.5% ratio for website._get_cached_values- 3507 hit 75.8% ratio for website.get_pricelist_available_cached> > We implemented CDN > e.g. The sentence "/web/image/company.logo" runs 10 queries > (prefetching fields, translating fields...) for each user opening the > page > After using CDN 0 queries are executed. But you just used Odoos standard CDN function or any specific measure here ?
> > We removed soft process that running with many users it is heavy > e.g. the random image that odoo assign when you create a res.partner > e.g. computed fields moved to cron Also here which analysis tools do you prefer to do so ? > > After that we detected concurrent update issues > It is hell for performance matter. > e.g. updating the same record at the same time from many users, > e.g. allowing pressing the same button too many times > e.g. sequences using no-gap in the main process, and so on > > After finishing all the hints of performance issues in the code and > database the next steps were to improve the hardware. > > Now, It supports 5k concurrent VUs doing checkout process (choose 2 > products, type address, payment card info and confirm) In what time frame ? > > So, Odoo will support those users but you will need to test, measure, > fix, test... > Stress testing tools helps us a lot here > python profilers and query analyzers too > > Maybe you are interest to watch our talk in SFO about: > - https://youtu.be/kiREZqg3JQg I will definitely have a look at this Thanks a lot for your extensive insight. Cheers from rainy Germany to sunny Mexico ;-) Frederik > > El mié, 28 abr 2021 a las 15:35, Frederik Kramer (< > frederik.kramer@initos.com>) escribió: > > Hi Folks, > > > > i have a project proposal on my desk that will ultimately end up in > > approx. 30 checkout transactions per minute. Has anybody got a > > project > > with that order volume done with a relatively recent version of > > Odoo > > and provide a good reference. Please bear in mind that the project > > ahead involves a similarly high volume of stock moves. > > > > Any suggestion / comment like (but not limited to) > > > > Odoo is the wrong technology for that, forget it, because.... > > > > over > > > > You can do it but, a), b), c) .... > > > > to > > > > If you really want that beast flying involve us over here because > > we > > are among the few ones that already did soemthing similar > > > > would be appreciated. > > > > From our load intensive projects at initOS i know that 10000 orders > > a > > day can be handled but almost 50000 is a different story and thats > > why > > i am asking the most experienced resource out there ;-) > > > > Thanks in advance for your replay > > > > Cheers Frederik > > > > > > -- > > Dr.-Ing. Frederik Kramer > > Geschäftsführer > > > > initOS GmbH > > An der Eisenbahn 1 > > 21224 Rosengarten > > > > Phone: +49 4105 56156-12 > > Fax: +49 4105 56156-10 > > Mobil: +49 179 3901819 > > > > Email: frederik.kramer@initos.com > > Web: www.initos.com > > > > Geschäftsführung: > > Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke > > > > Sitz der Gesellschaft: Rosengarten – Klecken > > Amtsgericht Tostedt, HRB 205226 > > Steuer-Nr: 15/200/53247 > > USt-IdNr.: DE815580155 > > > > _______________________________________________ > > Mailing-List: https://odoo-community.org/groups/contributors-15 > > Post to: mailto:contributors@odoo-community.org > > Unsubscribe: https://odoo-community.org/groups?unsubscribe > > -- Dr.-Ing. Frederik Kramer Geschäftsführer initOS GmbH An der Eisenbahn 1 21224 Rosengarten Phone: +49 4105 56156-12 Fax: +49 4105 56156-10 Mobil: +49 179 3901819 Email: frederik.kramer@initos.com Web: www.initos.com Geschäftsführung: Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke Sitz der Gesellschaft: Rosengarten – Klecken Amtsgericht Tostedt, HRB 205226 Steuer-Nr: 15/200/53247 USt-IdNr.: DE815580155
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
--Moisés López CalderónMobile: (+521) 477-752-22-30Twitter: @moylop260Twitter: @vauxoo
by Moisés López Calderón - 05:21 - 14 May 2021 -
Re: 30 Checkout Operations per Minute A job for Odoo or not ?
El jue, 6 de may. de 2021 a la(s) 05:02, Frederik Kramer (frederik.kramer@initos.com) escribió:Hi Moises, Am Freitag, den 30.04.2021, 16:02 +0000 schrieb Moises Lopez: > Hi Frederik > > A customer requested to support 5k concurrent sale.order (1 or 2 > lines of products) in Odoo. > The process that we followed were: test, measure, fix, test... haha, yes thats what works well if you have a customer that well endorses this "agile" and "pivoting" style of work. Apart from all economic considerations this is probably also something that isolated leads to quite solid solutions
That was our biggest measured customer (a recovered project) they know their KPI very well, on simpler customers, we try from a commercial perspective to explain exactly what Moises does (in a more extensive/commercial way) to try to teach them how to buy technology.At the end if you try let's say Shopify (the king of marketed e-commerce) with those numbers you mentioned or this ones they will offer you more **Enterprise** services (note the services not the software) because almost everybody works on this way even the order on which your usual customers make click on the website can change the outcome and bring performance/consistency problems, i.e: click twice a button in an slow process that by mistake land an specific day on your code).Then what's important IMHO is to clarify to the customer that set technically solid e-commerce is 30% software product 50% the right team.If this was not the case, we would have millions of amazon stores, Don't you think?> > The first time that we tested the checkout process supported 15 > concurrent virtual users (VUs) > The measure said that it had heavy custom functions in the same > transaction of checkout but it could be executed in background (cron, > queue) > and there were slow queries because of indexes issues or too many > queries executed because of python code did you use any specific profiling tools here? Or the ones that rcollet often proposes in his talks about the matter ?
Be careful here, they are a lot, and in my honest experience that's more the person using the tool than the tool itself.> > Note: Be careful with the base_automation if you can migrate them to > code it is better. Thanks for the hint, i think we experienced that also quite some times in the past and learned that lesson already ;-) > > After fix them the next iteration supported 400 VUs > So good but so far from the focus. > > The next iteration we found that there were methods that could be > using LRU-cache (e.g. prices, website that don't change) > Now we needed to check where the cache was clearing too frequently > and fix it. How did you measure LRU cache hit rates vs. code parts in a good way. Any tooling suggestion here ?
1. log-sql and a strong test setup that emulates the production environment is required.2. We use some SQL analyzer tools as well.But again, it is a matter of having the right team doing the job because it is not theoretical work, it is practice, test, read, practice, learn, fix, start again. and a little bit of good luck on some cases and being stubborn to solve the issues ;-)> > We implemented CDN > e.g. The sentence "/web/image/company.logo" runs 10 queries > (prefetching fields, translating fields...) for each user opening the > page > After using CDN 0 queries are executed. But you just used Odoos standard CDN function or any specific measure here ?
Standard! we simply use Odoo's CDN functionality.> > We removed soft process that running with many users it is heavy > e.g. the random image that odoo assign when you create a res.partner > e.g. computed fields moved to cron Also here which analysis tools do you prefer to do so ? > > After that we detected concurrent update issues > It is hell for performance matter. > e.g. updating the same record at the same time from many users, > e.g. allowing pressing the same button too many times > e.g. sequences using no-gap in the main process, and so on > > After finishing all the hints of performance issues in the code and > database the next steps were to improve the hardware. > > Now, It supports 5k concurrent VUs doing checkout process (choose 2 > products, type address, payment card info and confirm) In what time frame ?
With the real use case and in a production environment giving the problems more or less 60 days between First work and release into production ready for the action.> > So, Odoo will support those users but you will need to test, measure, > fix, test... > Stress testing tools helps us a lot here > python profilers and query analyzers too > > Maybe you are interest to watch our talk in SFO about: > - https://youtu.be/kiREZqg3JQg I will definitely have a look at this Thanks a lot for your extensive insight.
I summarize with:The team is the key to success here, every single piece in any e-commerce on this use case was given by different persons and led by Moisés because problems are in all layers: BD, Server, Sizing, Odoo, JS, OCA....It is not just one single pill!Cheers from rainy Germany to sunny Mexico ;-) Frederik > > El mié, 28 abr 2021 a las 15:35, Frederik Kramer (< > frederik.kramer@initos.com>) escribió: > > Hi Folks, > > > > i have a project proposal on my desk that will ultimately end up in > > approx. 30 checkout transactions per minute. Has anybody got a > > project > > with that order volume done with a relatively recent version of > > Odoo > > and provide a good reference. Please bear in mind that the project > > ahead involves a similarly high volume of stock moves. > > > > Any suggestion / comment like (but not limited to) > > > > Odoo is the wrong technology for that, forget it, because.... > > > > over > > > > You can do it but, a), b), c) .... > > > > to > > > > If you really want that beast flying involve us over here because > > we > > are among the few ones that already did soemthing similar > > > > would be appreciated. > > > > From our load intensive projects at initOS i know that 10000 orders > > a > > day can be handled but almost 50000 is a different story and thats > > why > > i am asking the most experienced resource out there ;-) > > > > Thanks in advance for your replay > > > > Cheers Frederik > > > > > > -- > > Dr.-Ing. Frederik Kramer > > Geschäftsführer > > > > initOS GmbH > > An der Eisenbahn 1 > > 21224 Rosengarten > > > > Phone: +49 4105 56156-12 > > Fax: +49 4105 56156-10 > > Mobil: +49 179 3901819 > > > > Email: frederik.kramer@initos.com > > Web: www.initos.com > > > > Geschäftsführung: > > Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke > > > > Sitz der Gesellschaft: Rosengarten – Klecken > > Amtsgericht Tostedt, HRB 205226 > > Steuer-Nr: 15/200/53247 > > USt-IdNr.: DE815580155 > > > > _______________________________________________ > > Mailing-List: https://odoo-community.org/groups/contributors-15 > > Post to: mailto:contributors@odoo-community.org > > Unsubscribe: https://odoo-community.org/groups?unsubscribe > > -- Dr.-Ing. Frederik Kramer Geschäftsführer initOS GmbH An der Eisenbahn 1 21224 Rosengarten Phone: +49 4105 56156-12 Fax: +49 4105 56156-10 Mobil: +49 179 3901819 Email: frederik.kramer@initos.com Web: www.initos.com Geschäftsführung: Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke Sitz der Gesellschaft: Rosengarten – Klecken Amtsgericht Tostedt, HRB 205226 Steuer-Nr: 15/200/53247 USt-IdNr.: DE815580155
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
--Nhomar G Hernández
Vauxoo | CEO
¡Construyamos algo genial!
Cel: +52 (477) 393.3942 | Telegram: nhomar | Twitter: @nhomarMéxico · Venezuela · Costa Rica · Perú
by Nhomar Hernández - 08:01 - 10 May 2021 -
Re: 30 Checkout Operations per Minute A job for Odoo or not ?
Hi Moises, Am Freitag, den 30.04.2021, 16:02 +0000 schrieb Moises Lopez: > Hi Frederik > > A customer requested to support 5k concurrent sale.order (1 or 2 > lines of products) in Odoo. > The process that we followed were: test, measure, fix, test... haha, yes thats what works well if you have a customer that well endorses this "agile" and "pivoting" style of work. Apart from all economic considerations this is probably also something that isolated leads to quite solid solutions > > The first time that we tested the checkout process supported 15 > concurrent virtual users (VUs) > The measure said that it had heavy custom functions in the same > transaction of checkout but it could be executed in background (cron, > queue) > and there were slow queries because of indexes issues or too many > queries executed because of python code did you use any specific profiling tools here? Or the ones that rcollet often proposes in his talks about the matter ? > > Note: Be careful with the base_automation if you can migrate them to > code it is better. Thanks for the hint, i think we experienced that also quite some times in the past and learned that lesson already ;-) > > After fix them the next iteration supported 400 VUs > So good but so far from the focus. > > The next iteration we found that there were methods that could be > using LRU-cache (e.g. prices, website that don't change) > Now we needed to check where the cache was clearing too frequently > and fix it. How did you measure LRU cache hit rates vs. code parts in a good way. Any tooling suggestion here ? > > We implemented CDN > e.g. The sentence "/web/image/company.logo" runs 10 queries > (prefetching fields, translating fields...) for each user opening the > page > After using CDN 0 queries are executed. But you just used Odoos standard CDN function or any specific measure here ? > > We removed soft process that running with many users it is heavy > e.g. the random image that odoo assign when you create a res.partner > e.g. computed fields moved to cron Also here which analysis tools do you prefer to do so ? > > After that we detected concurrent update issues > It is hell for performance matter. > e.g. updating the same record at the same time from many users, > e.g. allowing pressing the same button too many times > e.g. sequences using no-gap in the main process, and so on > > After finishing all the hints of performance issues in the code and > database the next steps were to improve the hardware. > > Now, It supports 5k concurrent VUs doing checkout process (choose 2 > products, type address, payment card info and confirm) In what time frame ? > > So, Odoo will support those users but you will need to test, measure, > fix, test... > Stress testing tools helps us a lot here > python profilers and query analyzers too > > Maybe you are interest to watch our talk in SFO about: > - https://youtu.be/kiREZqg3JQg I will definitely have a look at this Thanks a lot for your extensive insight. Cheers from rainy Germany to sunny Mexico ;-) Frederik > > El mié, 28 abr 2021 a las 15:35, Frederik Kramer (< > frederik.kramer@initos.com>) escribió: > > Hi Folks, > > > > i have a project proposal on my desk that will ultimately end up in > > approx. 30 checkout transactions per minute. Has anybody got a > > project > > with that order volume done with a relatively recent version of > > Odoo > > and provide a good reference. Please bear in mind that the project > > ahead involves a similarly high volume of stock moves. > > > > Any suggestion / comment like (but not limited to) > > > > Odoo is the wrong technology for that, forget it, because.... > > > > over > > > > You can do it but, a), b), c) .... > > > > to > > > > If you really want that beast flying involve us over here because > > we > > are among the few ones that already did soemthing similar > > > > would be appreciated. > > > > From our load intensive projects at initOS i know that 10000 orders > > a > > day can be handled but almost 50000 is a different story and thats > > why > > i am asking the most experienced resource out there ;-) > > > > Thanks in advance for your replay > > > > Cheers Frederik > > > > > > -- > > Dr.-Ing. Frederik Kramer > > Geschäftsführer > > > > initOS GmbH > > An der Eisenbahn 1 > > 21224 Rosengarten > > > > Phone: +49 4105 56156-12 > > Fax: +49 4105 56156-10 > > Mobil: +49 179 3901819 > > > > Email: frederik.kramer@initos.com > > Web: www.initos.com > > > > Geschäftsführung: > > Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke > > > > Sitz der Gesellschaft: Rosengarten – Klecken > > Amtsgericht Tostedt, HRB 205226 > > Steuer-Nr: 15/200/53247 > > USt-IdNr.: DE815580155 > > > > _______________________________________________ > > Mailing-List: https://odoo-community.org/groups/contributors-15 > > Post to: mailto:contributors@odoo-community.org > > Unsubscribe: https://odoo-community.org/groups?unsubscribe > > -- Dr.-Ing. Frederik Kramer Geschäftsführer initOS GmbH An der Eisenbahn 1 21224 Rosengarten Phone: +49 4105 56156-12 Fax: +49 4105 56156-10 Mobil: +49 179 3901819 Email: frederik.kramer@initos.com Web: www.initos.com Geschäftsführung: Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke Sitz der Gesellschaft: Rosengarten – Klecken Amtsgericht Tostedt, HRB 205226 Steuer-Nr: 15/200/53247 USt-IdNr.: DE815580155
by Frederik Kramer - 12:01 - 6 May 2021 -
Re: 30 Checkout Operations per Minute A job for Odoo or not ?
Hi FrederikA customer requested to support 5k concurrent sale.order (1 or 2 lines of products) in Odoo.The process that we followed were: test, measure, fix, test...The first time that we tested the checkout process supported 15 concurrent virtual users (VUs)The measure said that it had heavy custom functions in the same transaction of checkout but it could be executed in background (cron, queue)and there were slow queries because of indexes issues or too many queries executed because of python codeNote: Be careful with the base_automation if you can migrate them to code it is better.After fix them the next iteration supported 400 VUsSo good but so far from the focus.The next iteration we found that there were methods that could be using LRU-cache (e.g. prices, website that don't change)Now we needed to check where the cache was clearing too frequently and fix it.We implemented CDNe.g. The sentence "/web/image/company.logo" runs 10 queries (prefetching fields, translating fields...) for each user opening the pageAfter using CDN 0 queries are executed.We removed soft process that running with many users it is heavye.g. the random image that odoo assign when you create a res.partnere.g. computed fields moved to cronAfter that we detected concurrent update issuesIt is hell for performance matter.e.g. updating the same record at the same time from many users,e.g. allowing pressing the same button too many timese.g. sequences using no-gap in the main process, and so onAfter finishing all the hints of performance issues in the code and database the next steps were to improve the hardware.Now, It supports 5k concurrent VUs doing checkout process (choose 2 products, type address, payment card info and confirm)So, Odoo will support those users but you will need to test, measure, fix, test...Stress testing tools helps us a lot herepython profilers and query analyzers tooMaybe you are interest to watch our talk in SFO about:El mié, 28 abr 2021 a las 15:35, Frederik Kramer (<frederik.kramer@initos.com>) escribió:
Hi Folks, i have a project proposal on my desk that will ultimately end up in approx. 30 checkout transactions per minute. Has anybody got a project with that order volume done with a relatively recent version of Odoo and provide a good reference. Please bear in mind that the project ahead involves a similarly high volume of stock moves. Any suggestion / comment like (but not limited to) Odoo is the wrong technology for that, forget it, because.... over You can do it but, a), b), c) .... to If you really want that beast flying involve us over here because we are among the few ones that already did soemthing similar would be appreciated. From our load intensive projects at initOS i know that 10000 orders a day can be handled but almost 50000 is a different story and thats why i am asking the most experienced resource out there ;-) Thanks in advance for your replay Cheers Frederik -- Dr.-Ing. Frederik Kramer Geschäftsführer initOS GmbH An der Eisenbahn 1 21224 Rosengarten Phone: +49 4105 56156-12 Fax: +49 4105 56156-10 Mobil: +49 179 3901819 Email: frederik.kramer@initos.com Web: www.initos.com Geschäftsführung: Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke Sitz der Gesellschaft: Rosengarten – Klecken Amtsgericht Tostedt, HRB 205226 Steuer-Nr: 15/200/53247 USt-IdNr.: DE815580155_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
--Moisés López CalderónMobile: (+521) 477-752-22-30Twitter: @moylop260Twitter: @vauxoo
by Moisés López Calderón - 06:00 - 30 Apr 2021
