Search Postgresql Archives

Re: migrate off oracle data to postgres

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Though I am not very experienced in this, I am working on a migration project from Oracle to Postgres using ora2pg from around 5+ months. 
The source data from which I migrate is of size 64 GB and I use AWS environment and run ora2pg in parallel by specifying the parameters in ora2pg utility. I migrate 3 schemas containing around 192 tables data and it takes around 2.5+ hrs (in normal windows environment other then AWS it use to take more than 9 hrs for the same size of data) and then I do some checks like count of source objects, and count of rows in the source as well as destination data. This is to check if the data is imported as required, Later I do lot of other stuff to validate the data.

I am sure 1TeraByte of data will take a long time, again it depends on the environment and system configuration it is running.
Not must idea on MySQL workbench tool, sorry.


From: Pepe TD Vo <pepevo@xxxxxxxxx>
Sent: 31 March 2020 17:45
To: Pgsql-admin <pgsql-admin@xxxxxxxxxxxxxx>; Pgsql-general <pgsql-general@xxxxxxxxxxxxxx>; PgAdmin Support List <pgadmin-support@xxxxxxxxxxxxxxxxxxxx>; JC JC <shekharjchandra@xxxxxxxxxxxxx>
Subject: Re: migrate off oracle data to postgres
 
@JC - I do have ora2pg set up last year in test environment already and that what I suggested team using insert script to run in Postgres.  And I am not sure how long it would take to load? 1T data. Though if there's a tool liked MySQL workbench to pull the oracle export data pump and import into MySQL.

Bach-Nga

No one in this world is pure and perfect.  If you avoid people for their mistakes you will be alone. So judge less, love and forgive more.
To call him a dog hardly seems to do him justice though in as much as he had four legs, a tail, and barked, I admit he was, to all outward appearances. But to those who knew him well, he was a perfect gentleman (Hermione Gingold)

**Live simply **Love generously **Care deeply **Speak kindly.
*** Genuinely rich *** Faithful talent *** Sharing success


On Tuesday, March 31, 2020, 12:09:13 PM EDT, JC JC <shekharjchandra@xxxxxxxxxxxxx> wrote:


I am sure you might be referring this 

You need to create ora2pg conf file with TYPE COPY, and ALLOW <LIST OF TABLES separated by space> and remaining other attributes like 
ORACLE_DSN
ORACLE_PWD
ORACLE_USER

similarly 
PG_DSN
PG_USER
PG_PWD 
etc 
etc.

Cheers
jc


From: Pepe TD Vo <pepevo@xxxxxxxxx>
Sent: 31 March 2020 16:51
To: Pgsql-admin <pgsql-admin@xxxxxxxxxxxxxx>; Pgsql-general <pgsql-general@xxxxxxxxxxxxxx>; PgAdmin Support List <pgadmin-support@xxxxxxxxxxxxxxxxxxxx>; JC JC <shekharjchandra@xxxxxxxxxxxxx>
Subject: Re: migrate off oracle data to postgres
 
I have tables dddata definitions already.  Only need to move the data over off Oracle to Postgres

Bach-Nga

No one in this world is pure and perfect.  If you avoid people for their mistakes you will be alone. So judge less, love and forgive more.
To call him a dog hardly seems to do him justice though in as much as he had four legs, a tail, and barked, I admit he was, to all outward appearances. But to those who knew him well, he was a perfect gentleman (Hermione Gingold)

**Live simply **Love generously **Care deeply **Speak kindly.
*** Genuinely rich *** Faithful talent *** Sharing success


On Tuesday, March 31, 2020, 11:46:58 AM EDT, JC JC <shekharjchandra@xxxxxxxxxxxxx> wrote:


In my view ora2pg can do most of the stuff you requested.
ora2pg can be used to generate the DDL from oracle to postgres based on attributes set in ora2pg conf file (for example TYPE TABLE, will allow getting oracle DDL in an output file, TYPE COPY will bring the data). No doubt you need to set remaining attributes based on the requirement.

Export to CSV is not required, as mentioned TYPE COPY will do a load of data if you like it to do Direct Data Streaming.
As of my limited experience I never encountered any issue on the limit on data being transferred, but definitely the load time vary depending on the environment.

Regards
jc





From: Pepe TD Vo <pepevo@xxxxxxxxx>
Sent: 31 March 2020 15:15
To: Pgsql-admin <pgsql-admin@xxxxxxxxxxxxxx>; Pgsql-general <pgsql-general@xxxxxxxxxxxxxx>; PgAdmin Support List <pgadmin-support@xxxxxxxxxxxxxxxxxxxx>
Subject: migrate off oracle data to postgres
 
Hello,

I have a huge data on Oracle, would you please suggest how to migrate all data off Oracle to Postgres?  Do I need to export into csv and upload to postgres?  Ora2pg is limited data from oracle to postgresql right?


Bach-Nga

No one in this world is pure and perfect.  If you avoid people for their mistakes you will be alone. So judge less, love and forgive more.
To call him a dog hardly seems to do him justice though in as much as he had four legs, a tail, and barked, I admit he was, to all outward appearances. But to those who knew him well, he was a perfect gentleman (Hermione Gingold)

**Live simply **Love generously **Care deeply **Speak kindly.
*** Genuinely rich *** Faithful talent *** Sharing success

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Postgresql Jobs]     [Postgresql Admin]     [Postgresql Performance]     [Linux Clusters]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Postgresql & PHP]     [Yosemite]

  Powered by Linux