Yes, a Timescale hypertable with 500,000,000 rows of about 15 key/value pairs per record.
I'm not sure why there is both a gin & gist index on the hstore, or the merits of each.
Thanks....
\d t_reading_hstore_sec
Table "public.t_reading_hstore_sec"
Column | Type | Collation | Nullable | Default
------------+-----------------------------+-----------+----------+---------------------------------------------------
key | bigint | | not null | nextval('t_reading_hstore_sec_key_seq'::regclass)
timer | timestamp without time zone | | not null |
values_sec | hstore | | |
Indexes:
"t_reading_hstore_sec_pkey" PRIMARY KEY, btree (key, timer)
"t_reading_hstore_sec_timer_idx" btree (timer)
"t_reading_hstore_sec_timer_key" UNIQUE CONSTRAINT, btree (timer)
"t_reading_hstore_sec_values_idx_gin" gin (values_sec)
"t_reading_hstore_sec_values_idx_gist" gist (values_sec)
Table "public.t_reading_hstore_sec"
Column | Type | Collation | Nullable | Default
------------+-----------------------------+-----------+----------+---------------------------------------------------
key | bigint | | not null | nextval('t_reading_hstore_sec_key_seq'::regclass)
timer | timestamp without time zone | | not null |
values_sec | hstore | | |
Indexes:
"t_reading_hstore_sec_pkey" PRIMARY KEY, btree (key, timer)
"t_reading_hstore_sec_timer_idx" btree (timer)
"t_reading_hstore_sec_timer_key" UNIQUE CONSTRAINT, btree (timer)
"t_reading_hstore_sec_values_idx_gin" gin (values_sec)
"t_reading_hstore_sec_values_idx_gist" gist (values_sec)
On Wednesday, January 22, 2025 at 06:34:38 AM GMT+13, Adrian Klaver <adrian.klaver@xxxxxxxxxxx> wrote:
On 1/19/25 12:09, Brent Wood wrote:
> Thanks for the replies, appreciated...
>
> My current solution is:
>
> /select trip_code,/
> / station_no,/
> / timer_sec + interval '12 hour' as NZST,/
> / timer_sec as utc,/
> / hstore_to_json(string_agg(values_sec::text, ', ')::hstore)
> as values_sec/
> / from (select '$TRIP' as trip_code,/
> / $STATION as station_no,/
> / date_trunc('second', timer) as timer_sec,/
> / values_sec/
> / from t_reading_hstore_sec/
> / where timer >= '$ISO_S'::timestamp - interval '12 hour'/
> / and timer <= '$ISO_F'::timestamp - interval '12 hour') as foo/
> /group by timer_sec, trip_code, station_no;/
>
> Convert the hstore to text, aggregate the text with string_agg(),
> convert back to hstore (which seems to remove duplicate keys, OK for my
> purpose)
To be clear values_sec in t_reading_hstore_sec is the hstore field?
If so what is it's structure?
> and group by timer truncated to whole seconds. I also provide UTC &
> local timezone times for each set of readings. It is run in a bash
> script which passes the trip & station values to the query, as well as
> the start/finish times as ISO format strings.
>
> The output is going to a Sqlite3 (Spatialite) database, which does not
> have hstore, or all the hstore functionality that Postgres has, but does
> have a json datatype which is adequate for our purposes, hence the
> hstore_to_json in the query.
>
>
> Thanks again,
>
> Brent Wood
>
--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx
> Thanks for the replies, appreciated...
>
> My current solution is:
>
> /select trip_code,/
> / station_no,/
> / timer_sec + interval '12 hour' as NZST,/
> / timer_sec as utc,/
> / hstore_to_json(string_agg(values_sec::text, ', ')::hstore)
> as values_sec/
> / from (select '$TRIP' as trip_code,/
> / $STATION as station_no,/
> / date_trunc('second', timer) as timer_sec,/
> / values_sec/
> / from t_reading_hstore_sec/
> / where timer >= '$ISO_S'::timestamp - interval '12 hour'/
> / and timer <= '$ISO_F'::timestamp - interval '12 hour') as foo/
> /group by timer_sec, trip_code, station_no;/
>
> Convert the hstore to text, aggregate the text with string_agg(),
> convert back to hstore (which seems to remove duplicate keys, OK for my
> purpose)
To be clear values_sec in t_reading_hstore_sec is the hstore field?
If so what is it's structure?
> and group by timer truncated to whole seconds. I also provide UTC &
> local timezone times for each set of readings. It is run in a bash
> script which passes the trip & station values to the query, as well as
> the start/finish times as ISO format strings.
>
> The output is going to a Sqlite3 (Spatialite) database, which does not
> have hstore, or all the hstore functionality that Postgres has, but does
> have a json datatype which is adequate for our purposes, hence the
> hstore_to_json in the query.
>
>
> Thanks again,
>
> Brent Wood
>
--
Adrian Klaver
adrian.klaver@xxxxxxxxxxx
