Archives
Contributors
-
RFQ for OCA Days 2024 Video Post Production
Hello OCA Contributors,I hope this finds you all well.We are getting the final details ready for OCA Days 2024.If you haven't registered yet, there is still time, you can do so here.I'm sharing the RFQ for Video Post Production of the recorded talks - have you got a digital superstar in your company who could help? Please share it around.Deadline for submission for the RFQ is 18th September 2024.You can find out more on our RFQ process page.If you have any questions please get in touch.Rebecca--Rebecca GellatlyGeneral SecretaryOdoo Community Association
by Rebecca Gellatly (OCA) - 01:16 - 28 Aug 2024 -
Re: odoo-community.org has a new home
thanks :)
From: Rafael Blasco <notifications@odoo-community.org>
Sent: Wednesday, August 28, 2024 12:07 PM
To: Contributors <contributors@odoo-community.org>
Subject: RE: odoo-community.org has a new homeThank you very much Stéphane!
De: Stéphane Bidoul <notifications@odoo-community.org>
Enviado el: jueves, 22 de agosto de 2024 0:08
Para: Contributors <contributors@odoo-community.org>
Asunto: odoo-community.org has a new homeHi everyone,
https://odoo-community.org has found a new home on a more powerful server.
Everything should work as before, just faster. Let me know if you notice anything is off.
Best regards,
-Stéphane
--
Stéphane Bidoul
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Manish Bohra. - 09:51 - 28 Aug 2024 -
RE: odoo-community.org has a new home
Thank you very much Stéphane!
De: Stéphane Bidoul <notifications@odoo-community.org>
Enviado el: jueves, 22 de agosto de 2024 0:08
Para: Contributors <contributors@odoo-community.org>
Asunto: odoo-community.org has a new homeHi everyone,
https://odoo-community.org has found a new home on a more powerful server.
Everything should work as before, just faster. Let me know if you notice anything is off.
Best regards,
-Stéphane
--
Stéphane Bidoul
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Rafael Blasco (Moduon) - 08:36 - 28 Aug 2024 -
Re: odoo-community.org has a new home
Super cool! Thank you Stéphane! :)On Thu, Aug 22, 2024 at 12:07 AM Stéphane Bidoul <notifications@odoo-community.org> wrote:Hi everyone,https://odoo-community.org has found a new home on a more powerful server.Everything should work as before, just faster. Let me know if you notice anything is off.Best regards,-Stéphane--Stéphane Bidoul_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
--Simone OrsiFull stack Python web developer, Odoo specialist, Odoo Community Board Member, in love with open source.
by Simone Orsi - 08:26 - 27 Aug 2024 -
Re: odoo-community.org has a new home
Very awesome and very appreciated!On Tue, 27 Aug 2024 at 02:38, Francesco Ballerini <notifications@odoo-community.org> wrote:Nice! Thanks--Francesco BalleriniIl giorno lun 26 ago 2024 alle ore 14:18 Antonio M. Vigliotti <notifications@odoo-community.org> ha scritto:
Great job!
Antonio M. Vigliotti
Mobile (+39) 342.8740910
Il 26/08/2024 10:02, Virginie Dewulf ha scritto:
Many thanks, it changes a lot on the backend side as well :)
Le lun. 26 août 2024 à 08:43, David Beal <notifications@odoo-community.org> a écrit :
Le lun. 26 août 2024 à 08:32, Alexandre Fayolle <notifications@odoo-community.org> a écrit :
Thanks :) Alexandre On 22/08/2024 00:07, Stéphane Bidoul wrote: > Hi everyone, > > https://odoo-community.org <https://odoo-community.org> has found a new > home on a more powerful server. > > Everything should work as before, just faster. Let me know if you notice > anything is off. > > Best regards, > > -Stéphane > > -- > Stéphane Bidoul > http://acsone.eu/ <http://acsone.eu/> > > _______________________________________________ > Mailing-List: https://odoo-community.org/groups/contributors-15 > <https://odoo-community.org/groups/contributors-15> > Post to: mailto:contributors@odoo-community.org > Unsubscribe: https://odoo-community.org/groups?unsubscribe > <https://odoo-community.org/groups?unsubscribe> > -- Alexandre Fayolle Senior Software Engineer Camptocamp France SAS 18 rue du Lac Saint André 73 370 Le Bourget-du-Lac France http://www.camptocamp.com
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
--Rebecca GellatlyGeneral SecretaryOdoo Community Association
by Rebecca Gellatly (OCA) - 03:53 - 27 Aug 2024 -
Re: odoo-community.org has a new home
Nice! Thanks--Francesco BalleriniIl giorno lun 26 ago 2024 alle ore 14:18 Antonio M. Vigliotti <notifications@odoo-community.org> ha scritto:
Great job!
Antonio M. Vigliotti
Mobile (+39) 342.8740910
Il 26/08/2024 10:02, Virginie Dewulf ha scritto:
Many thanks, it changes a lot on the backend side as well :)
Le lun. 26 août 2024 à 08:43, David Beal <notifications@odoo-community.org> a écrit :
Le lun. 26 août 2024 à 08:32, Alexandre Fayolle <notifications@odoo-community.org> a écrit :
Thanks :) Alexandre On 22/08/2024 00:07, Stéphane Bidoul wrote: > Hi everyone, > > https://odoo-community.org <https://odoo-community.org> has found a new > home on a more powerful server. > > Everything should work as before, just faster. Let me know if you notice > anything is off. > > Best regards, > > -Stéphane > > -- > Stéphane Bidoul > http://acsone.eu/ <http://acsone.eu/> > > _______________________________________________ > Mailing-List: https://odoo-community.org/groups/contributors-15 > <https://odoo-community.org/groups/contributors-15> > Post to: mailto:contributors@odoo-community.org > Unsubscribe: https://odoo-community.org/groups?unsubscribe > <https://odoo-community.org/groups?unsubscribe> > -- Alexandre Fayolle Senior Software Engineer Camptocamp France SAS 18 rue du Lac Saint André 73 370 Le Bourget-du-Lac France http://www.camptocamp.com
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Francesco Ballerini - 04:36 - 26 Aug 2024 -
Re: odoo-community.org has a new home
Great job!
Antonio M. Vigliotti
Mobile (+39) 342.8740910
Il 26/08/2024 10:02, Virginie Dewulf ha scritto:
Many thanks, it changes a lot on the backend side as well :)
Le lun. 26 août 2024 à 08:43, David Beal <notifications@odoo-community.org> a écrit :
Le lun. 26 août 2024 à 08:32, Alexandre Fayolle <notifications@odoo-community.org> a écrit :
Thanks :) Alexandre On 22/08/2024 00:07, Stéphane Bidoul wrote: > Hi everyone, > > https://odoo-community.org <https://odoo-community.org> has found a new > home on a more powerful server. > > Everything should work as before, just faster. Let me know if you notice > anything is off. > > Best regards, > > -Stéphane > > -- > Stéphane Bidoul > http://acsone.eu/ <http://acsone.eu/> > > _______________________________________________ > Mailing-List: https://odoo-community.org/groups/contributors-15 > <https://odoo-community.org/groups/contributors-15> > Post to: mailto:contributors@odoo-community.org > Unsubscribe: https://odoo-community.org/groups?unsubscribe > <https://odoo-community.org/groups?unsubscribe> > -- Alexandre Fayolle Senior Software Engineer Camptocamp France SAS 18 rue du Lac Saint André 73 370 Le Bourget-du-Lac France http://www.camptocamp.com
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Antonio M. Vigliotti - 02:16 - 26 Aug 2024 -
Re: odoo-community.org has a new home
Many thanks, it changes a lot on the backend side as well :)Le lun. 26 août 2024 à 08:43, David Beal <notifications@odoo-community.org> a écrit :Le lun. 26 août 2024 à 08:32, Alexandre Fayolle <notifications@odoo-community.org> a écrit :Thanks :) Alexandre On 22/08/2024 00:07, Stéphane Bidoul wrote: > Hi everyone, > > https://odoo-community.org <https://odoo-community.org> has found a new > home on a more powerful server. > > Everything should work as before, just faster. Let me know if you notice > anything is off. > > Best regards, > > -Stéphane > > -- > Stéphane Bidoul > http://acsone.eu/ <http://acsone.eu/> > > _______________________________________________ > Mailing-List: https://odoo-community.org/groups/contributors-15 > <https://odoo-community.org/groups/contributors-15> > Post to: mailto:contributors@odoo-community.org > Unsubscribe: https://odoo-community.org/groups?unsubscribe > <https://odoo-community.org/groups?unsubscribe> > -- Alexandre Fayolle Senior Software Engineer Camptocamp France SAS 18 rue du Lac Saint André 73 370 Le Bourget-du-Lac France http://www.camptocamp.com
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Virginie Dewulf - 10:00 - 26 Aug 2024 -
Re: odoo-community.org has a new home
Le lun. 26 août 2024 à 08:32, Alexandre Fayolle <notifications@odoo-community.org> a écrit :Thanks :) Alexandre On 22/08/2024 00:07, Stéphane Bidoul wrote: > Hi everyone, > > https://odoo-community.org <https://odoo-community.org> has found a new > home on a more powerful server. > > Everything should work as before, just faster. Let me know if you notice > anything is off. > > Best regards, > > -Stéphane > > -- > Stéphane Bidoul > http://acsone.eu/ <http://acsone.eu/> > > _______________________________________________ > Mailing-List: https://odoo-community.org/groups/contributors-15 > <https://odoo-community.org/groups/contributors-15> > Post to: mailto:contributors@odoo-community.org > Unsubscribe: https://odoo-community.org/groups?unsubscribe > <https://odoo-community.org/groups?unsubscribe> > -- Alexandre Fayolle Senior Software Engineer Camptocamp France SAS 18 rue du Lac Saint André 73 370 Le Bourget-du-Lac France http://www.camptocamp.com
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by David BEAL - 08:42 - 26 Aug 2024 -
Re: odoo-community.org has a new home
Thanks :) Alexandre On 22/08/2024 00:07, Stéphane Bidoul wrote: > Hi everyone, > > https://odoo-community.org <https://odoo-community.org> has found a new > home on a more powerful server. > > Everything should work as before, just faster. Let me know if you notice > anything is off. > > Best regards, > > -Stéphane > > -- > Stéphane Bidoul > http://acsone.eu/ <http://acsone.eu/> > > _______________________________________________ > Mailing-List: https://odoo-community.org/groups/contributors-15 > <https://odoo-community.org/groups/contributors-15> > Post to: mailto:contributors@odoo-community.org > Unsubscribe: https://odoo-community.org/groups?unsubscribe > <https://odoo-community.org/groups?unsubscribe> > -- Alexandre Fayolle Senior Software Engineer Camptocamp France SAS 18 rue du Lac Saint André 73 370 Le Bourget-du-Lac France http://www.camptocamp.com
by Alexandre Fayolle - 08:31 - 26 Aug 2024 -
Re: Odoo Experience Exhibition 2024, Visitors Data
YES INTRESTED
Best regards,
Morgane Goujon
Développeuse Logiciel
Scalizer
morgane@scalizer.fr 
www.scalizer.fr 
13 rue Crucy, 44000 Nantes Le jeu. 22 août 2024 à 10:27, Michel Stroom <notifications@odoo-community.org> a écrit :YES INTERESTEDThanks for sharing ElizaBest regards,Michel StroomNetherlands--Office Everywhere
t: +31 6 53360677
e: mstroom@office-everywhere.com
w: Office-Everywhere.comOffice
Ramstraat 31-33
3581 HD Utrecht
On 12 Aug 2024, at 16:57, Amelia <notifications@odoo-community.org> wrote:Hope you are doing well !!
To help you, we are offering detailed visitor data collected for the exhibition
We are following up to confirm if you are interested in acquiring the Visitors/Attendees List.
Event Name : Odoo Experience Exhibition 2024
Date : 02 - 04 Oct 2024
Location : Brussels Expo - Exhibition Center, Brussels, Belgium
Counts : 4,300
If you are interested in acquiring the Exact list, we can Send you the Discounted cost and real additional details.
Each record of the list contains: Contact Name, Email Address, Company Name, URL/Website, Phone No, Title/Designation.
Exhibitors who utilized our visitor data in previous events have seen a significant increase in lead conversion rates and tailored their follow-up strategies effectively. By analyzing this data, you can better understand your audience, refine your marketing approach, and follow up with highly qualified leads.
We are looking forward to hearing from you.
If you interested to take visitors list, please reply to me has ” YES INTRESTED “
Thanks & Regards,
Eliza
Market Analyst.
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by morgane - 12:31 - 22 Aug 2024 -
Re: Odoo Experience Exhibition 2024, Visitors Data
YES INTERESTEDThanks for sharing ElizaBest regards,Michel StroomNetherlands--Office Everywhere
t: +31 6 53360677
e: mstroom@office-everywhere.com
w: Office-Everywhere.comOffice
Ramstraat 31-33
3581 HD Utrecht
On 12 Aug 2024, at 16:57, Amelia <notifications@odoo-community.org> wrote:Hope you are doing well !!
To help you, we are offering detailed visitor data collected for the exhibition
We are following up to confirm if you are interested in acquiring the Visitors/Attendees List.
Event Name : Odoo Experience Exhibition 2024
Date : 02 - 04 Oct 2024
Location : Brussels Expo - Exhibition Center, Brussels, Belgium
Counts : 4,300
If you are interested in acquiring the Exact list, we can Send you the Discounted cost and real additional details.
Each record of the list contains: Contact Name, Email Address, Company Name, URL/Website, Phone No, Title/Designation.
Exhibitors who utilized our visitor data in previous events have seen a significant increase in lead conversion rates and tailored their follow-up strategies effectively. By analyzing this data, you can better understand your audience, refine your marketing approach, and follow up with highly qualified leads.
We are looking forward to hearing from you.
If you interested to take visitors list, please reply to me has ” YES INTRESTED “
Thanks & Regards,
Eliza
Market Analyst.
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Michel Stroom - 10:26 - 22 Aug 2024 -
Re: odoo-community.org has a new home
Thank you very much! I don't think it's related to this migration but the download button of the shop product page doesn't seem to work (nothing gets downloaded). Not that I'm troubled by it. Just reporting.--田代祥隆 Yoshi Tashiroコタエル株式会社 / QuartileOn Thu, Aug 22, 2024 at 7:07 AM Stéphane Bidoul <notifications@odoo-community.org> wrote:Hi everyone,https://odoo-community.org has found a new home on a more powerful server.Everything should work as before, just faster. Let me know if you notice anything is off.Best regards,-Stéphane--Stéphane Bidoul_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Yoshi Tashiro. - 03:25 - 22 Aug 2024 -
odoo-community.org has a new home
Hi everyone,https://odoo-community.org has found a new home on a more powerful server.Everything should work as before, just faster. Let me know if you notice anything is off.Best regards,-Stéphane--Stéphane Bidoul
by Stéphane Bidoul - 12:06 - 22 Aug 2024 -
Re: Test
Success again!Regards,Technology Services @ www.serpentcs.comBusiness Solutions @ www.serpentcs.inEnterprise Mobile Apps @ www.odooonline.comQuality Assurance @ www.odooqa.comSAP Hana @ www.prozone-tech.comPortal & DMS @ www.alfray.inOn Wed, 21 Aug 2024 at 5:28 PM, Stéphane Bidoul <notifications@odoo-community.org> wrote:And testing again from the other side.On Tue, Aug 20, 2024 at 11:12 PM Pierre Verkest <notifications@odoo-community.org> wrote:It's wirking !Le mar. 20 août 2024 à 22:43, Stéphane Bidoul <notifications@odoo-community.org> a écrit :This is a test message, preparing for the migration of the OCA Odoo instance to a new server.-sbi_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Jay Vora - 11:41 - 21 Aug 2024 -
Re: Test
And testing again from the other side.On Tue, Aug 20, 2024 at 11:12 PM Pierre Verkest <notifications@odoo-community.org> wrote:It's wirking !Le mar. 20 août 2024 à 22:43, Stéphane Bidoul <notifications@odoo-community.org> a écrit :This is a test message, preparing for the migration of the OCA Odoo instance to a new server.-sbi
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Stéphane Bidoul - 11:26 - 21 Aug 2024 -
Re: Large Data Files
Nice to see pandas run fast.For those for those that want run very very ... very fast consider to use polarsLe mer. 21 août 2024 à 01:57, Graeme Gellatly <notifications@odoo-community.org> a écrit :Queue job and batching can work. It is commonplace. But if it is CPU/Memory then honestly, after optimising what you can within the framework (e.g. as per Holger) a lot of the time you just get away with running a seperate worker on a separate port for long running jobs and set the limits/timeouts high. That is how a lot of people deploy cron workers these days and in older Odoo we used to have to do it to run financial reports and seemingly again now. 30,000 simple records is not so much.There may also be some db tuning you can do around WAL files, checkpoints etc if they get in the way.On Wed, Aug 21, 2024 at 9:57 AM Jerôme Dewandre <notifications@odoo-community.org> wrote:Hello,Thank you very much for your quick responses :)
Tom Blauwendraat: I am running on v16Holger Brunn: adapting the script with .with_context(tracking_disable=True) to Disable email notification divides the running time by at least 4
Goran Sunjka: It is indeed an interesting idea, I was wondering if I could store a hash of the row in Postgres to check if an existing record was updated to separate "create" and "update" action
Daniel Reis: This is indeed the problem I encountered.
Thank you all for your replies, it helps a lot :)JérômeOn Tue, Aug 20, 2024 at 7:47 PM Daniel Reis <notifications@odoo-community.org> wrote:I would expect this code to just abort for a non trivial quantity of records.
The reason why is that this is a single worker doing a single database transaction.
So the worker process will probably hit the time and CPU limits and be killed, and no records would be saved because of a transaction rollback.
And if you increase those limits a lot, you will probably cause long table locks on the database, and hurt other users and processes.
Going direct to the database can work if the data is pretty simple.
It can work but it can also be a can of worms.
One approach is to have an incremental approach to the data loading.
In the past I have used external ETL tools or scripts to do this.
Keeping it inside Odoo, one of the tools that can help is the Job Queue, possibly along with something like base_import_async:
https://github.com/OCA/queue/tree/16.0/base_import_async
Thanks
--
DANIEL REIS
MANAGING PARTNERMeet with me.
M: +351 919 991 307
E: dreis@OpenSourceIntegrators.com
A: Avenida da República 3000, Estoril Office Center, 2649-517 Cascais![[Logo OpenSourceIntegrators.com]](https://www.opensourceintegrators.com/images/opensource-logo-2022.svg)
On 20/08/2024 16:32, Jerôme Dewandre wrote:
Hello,
I am currently working on a syncro with a legacy system (adesoft) containing a large amount of data that must be synchronized on a daily basis (such as meetings).
It seems everything starts getting slow when I import 30.000 records with the conventional "create()" method.
I suppose the ORM might be an issue here. Potential workaround:
1. Bypass the ORM to create a record with self.env.cr.execute (but if I want to delete them I will also need a custom query)2. Bypass the ORM with stored procedures (https://www.postgresql.org/docs/current/sql-createprocedure.html)3. Increase the CPU/RAM/Worker nodes4. Some better ideas?
What would be the best way to go?
A piece of my current test (df is a pandas dataframe containing the new events):
@api.modeldef create_events_from_df(self, df):Event = self.env['event.event']events_data = []for _, row in df.iterrows():event_data = {'location': row['location'],'name': row['name'],'date_begin': row['date_begin'],'date_end': row['date_end'],}events_data.append(event_data)# Create all events in a single batchEvent.create(events_data)
Thanks in advance if you read this, and thanks again if you replied :)
Jérôme_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by David BEAL - 09:16 - 21 Aug 2024 -
Re: Large Data Files
Queue job and batching can work. It is commonplace. But if it is CPU/Memory then honestly, after optimising what you can within the framework (e.g. as per Holger) a lot of the time you just get away with running a seperate worker on a separate port for long running jobs and set the limits/timeouts high. That is how a lot of people deploy cron workers these days and in older Odoo we used to have to do it to run financial reports and seemingly again now. 30,000 simple records is not so much.There may also be some db tuning you can do around WAL files, checkpoints etc if they get in the way.On Wed, Aug 21, 2024 at 9:57 AM Jerôme Dewandre <notifications@odoo-community.org> wrote:Hello,Thank you very much for your quick responses :)
Tom Blauwendraat: I am running on v16Holger Brunn: adapting the script with .with_context(tracking_disable=True) to Disable email notification divides the running time by at least 4
Goran Sunjka: It is indeed an interesting idea, I was wondering if I could store a hash of the row in Postgres to check if an existing record was updated to separate "create" and "update" action
Daniel Reis: This is indeed the problem I encountered.
Thank you all for your replies, it helps a lot :)JérômeOn Tue, Aug 20, 2024 at 7:47 PM Daniel Reis <notifications@odoo-community.org> wrote:I would expect this code to just abort for a non trivial quantity of records.
The reason why is that this is a single worker doing a single database transaction.
So the worker process will probably hit the time and CPU limits and be killed, and no records would be saved because of a transaction rollback.
And if you increase those limits a lot, you will probably cause long table locks on the database, and hurt other users and processes.
Going direct to the database can work if the data is pretty simple.
It can work but it can also be a can of worms.
One approach is to have an incremental approach to the data loading.
In the past I have used external ETL tools or scripts to do this.
Keeping it inside Odoo, one of the tools that can help is the Job Queue, possibly along with something like base_import_async:
https://github.com/OCA/queue/tree/16.0/base_import_async
Thanks
--
DANIEL REIS
MANAGING PARTNERMeet with me.
M: +351 919 991 307
E: dreis@OpenSourceIntegrators.com
A: Avenida da República 3000, Estoril Office Center, 2649-517 Cascais
On 20/08/2024 16:32, Jerôme Dewandre wrote:
Hello,
I am currently working on a syncro with a legacy system (adesoft) containing a large amount of data that must be synchronized on a daily basis (such as meetings).
It seems everything starts getting slow when I import 30.000 records with the conventional "create()" method.
I suppose the ORM might be an issue here. Potential workaround:
1. Bypass the ORM to create a record with self.env.cr.execute (but if I want to delete them I will also need a custom query)2. Bypass the ORM with stored procedures (https://www.postgresql.org/docs/current/sql-createprocedure.html)3. Increase the CPU/RAM/Worker nodes4. Some better ideas?
What would be the best way to go?
A piece of my current test (df is a pandas dataframe containing the new events):
@api.modeldef create_events_from_df(self, df):Event = self.env['event.event']events_data = []for _, row in df.iterrows():event_data = {'location': row['location'],'name': row['name'],'date_begin': row['date_begin'],'date_end': row['date_end'],}events_data.append(event_data)# Create all events in a single batchEvent.create(events_data)
Thanks in advance if you read this, and thanks again if you replied :)
Jérôme_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by "Graeme Gellatly" <graeme@moahub.nz> - 01:56 - 21 Aug 2024 -
Re: Large Data Files
Hello,Thank you very much for your quick responses :)
Tom Blauwendraat: I am running on v16Holger Brunn: adapting the script with .with_context(tracking_disable=True) to Disable email notification divides the running time by at least 4
Goran Sunjka: It is indeed an interesting idea, I was wondering if I could store a hash of the row in Postgres to check if an existing record was updated to separate "create" and "update" action
Daniel Reis: This is indeed the problem I encountered.
Thank you all for your replies, it helps a lot :)JérômeOn Tue, Aug 20, 2024 at 7:47 PM Daniel Reis <notifications@odoo-community.org> wrote:I would expect this code to just abort for a non trivial quantity of records.
The reason why is that this is a single worker doing a single database transaction.
So the worker process will probably hit the time and CPU limits and be killed, and no records would be saved because of a transaction rollback.
And if you increase those limits a lot, you will probably cause long table locks on the database, and hurt other users and processes.
Going direct to the database can work if the data is pretty simple.
It can work but it can also be a can of worms.
One approach is to have an incremental approach to the data loading.
In the past I have used external ETL tools or scripts to do this.
Keeping it inside Odoo, one of the tools that can help is the Job Queue, possibly along with something like base_import_async:
https://github.com/OCA/queue/tree/16.0/base_import_async
Thanks
--
DANIEL REIS
MANAGING PARTNERMeet with me.
M: +351 919 991 307
E: dreis@OpenSourceIntegrators.com
A: Avenida da República 3000, Estoril Office Center, 2649-517 Cascais
On 20/08/2024 16:32, Jerôme Dewandre wrote:
Hello,
I am currently working on a syncro with a legacy system (adesoft) containing a large amount of data that must be synchronized on a daily basis (such as meetings).
It seems everything starts getting slow when I import 30.000 records with the conventional "create()" method.
I suppose the ORM might be an issue here. Potential workaround:
1. Bypass the ORM to create a record with self.env.cr.execute (but if I want to delete them I will also need a custom query)2. Bypass the ORM with stored procedures (https://www.postgresql.org/docs/current/sql-createprocedure.html)3. Increase the CPU/RAM/Worker nodes4. Some better ideas?
What would be the best way to go?
A piece of my current test (df is a pandas dataframe containing the new events):
@api.modeldef create_events_from_df(self, df):Event = self.env['event.event']events_data = []for _, row in df.iterrows():event_data = {'location': row['location'],'name': row['name'],'date_begin': row['date_begin'],'date_end': row['date_end'],}events_data.append(event_data)# Create all events in a single batchEvent.create(events_data)
Thanks in advance if you read this, and thanks again if you replied :)
Jérôme_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by "Jerôme Dewandre" <jerome.dewandre.mail@gmail.com> - 11:56 - 20 Aug 2024 -
Re: Test
It's wirking !Le mar. 20 août 2024 à 22:43, Stéphane Bidoul <notifications@odoo-community.org> a écrit :This is a test message, preparing for the migration of the OCA Odoo instance to a new server.-sbi_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
by Pierre Verkest - 11:11 - 20 Aug 2024
