Re: Data replication to PostgreSQL

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Scott,

Thank you for the inputs. We are using SOQL api calls to get the data from Salesforce back-end tables and replicating it to Postgres. One challenge we face is with metadata changes to Salesforce tables. They keep dynamically changing with the business requirements. Do we have any process to capture the DDL changes and replicate it Postgres?
Also, salesforce API calls does not support "SELECT * FROM <table>". Instead, we had to mention each field/column and select the data.


Thank you,
Ram

On Thu, Dec 29, 2016 at 10:58 PM, Scott Mead <scottm@xxxxxxxxxxx> wrote:

> On Dec 29, 2016, at 09:35, Ram DBA <postgresdbas@xxxxxxxxx> wrote:
>
> Hi team,
>
> We have an use case to copy the data from Salesforce to PostgreSQL environment but because of DDL changes that can happen to tables from Salesforce application side, replication would eventually fail. Do we have any open source tools that can also capture the DDL changes and replicate to PostgreSQL?

Are you using an export tool provided by salesforce, or a custom tool against their APIs?

  Either way, you don't have access to their database, you should be dumping the data to a CSV and importing the CSV to postgres. It's been a while since I've monkeyed with it, There may be a new way of doing this, but I doubt it. As long as your export tool always creates a csv in the right format, you'll be fine. If you aren't ordering your columns or are using *, you'll never get a good read. Use the toon or API to create a consistent file each time in some standard format and you'll be as okay as possible.

--
Scott Mead via mobile
Skype  : scottm.openscg
Gtalk    : scottm@xxxxxxxxxxx

>
> Thank you in advance,
> Ram


[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux