Good Morning,
This is probably not the best place to ask this, but I'd figure I would give it a shot.
We are slowly moving over to a hybrid nosql model, and many of our large sets of data are being stored in json format, specifically using the jsonb data type.
The issue is that the etl tool that I've developed to parse this data into a datawarehouse is very manual and pretty cumbersome - essentially using the jsonb postgresql query commands and cron or pgagent. It's unable to effectively traverse the element levels, and really isn't very good at handling dynamic changes.
I am wondering what people use for a tool to do this work. We are a smaller shop and do not need a large solution. Would prefer something that integrates well with postgresql and can be easily modified to change data mappings, and possibly support additional filetypes outside of json.
I am wondering what people use for a tool to do this work. We are a smaller shop and do not need a large solution. Would prefer something that integrates well with postgresql and can be easily modified to change data mappings, and possibly support additional filetypes outside of json.
The end result would be a table or set of tables that can be ingested by a visualization tool like Tableau or PowerBI.
Thanks!
-Jayson
--
Jayson Hreczuck | Principal Database Administrator
C: +1 617.571.9861
130 Turner St, Ste 530, Waltham, MA 02453