Re: a question about oddities in the data directory

[Melvin Davidson <melvin6925@xxxxxxxxx>]

The directory "base" contains filenames(numbers) that correspond to the
oid of the postgreSQL databases. Those databases have subdirectories
that contain the filenames(numbers) that correspond to the oid of objects
(sequences, tables, etc) in those databases.

You may find the attached queries helpul in navigating those directories

On Mon, Nov 27, 2017 at 10:27 AM, Tomas Vondra <tomas.vondra@xxxxxxxxxxxxxxx> wrote:

Hi,

I think you need to read this:

https://www.postgresql.org/docs/current/static/storage-file-layout.html

On 11/27/2017 04:19 PM, Martin Mueller wrote:
> Apologies if I asked this question before.
>
> I’m a newcomer to Postgres, having migrated from Mysql.  Information
> about tables seems harder to get at in Postgres. That data directory
> on my machine is suspiciously large—well over 100 GB.  The directory
> Postgres/var-9.5/base/ contains a number of subdirectories with
> Arabic numerals.

Each of those directories stores data for a single database. Each object
(including databases) has a 32-bit identifier, mapping it to object on
the filesystem.

databases: SELECT oid, datname FROM pg_database;
tables etc.: SELECT relfilenode, relname FROM pg_class;

> Directory 16385 has a subdirectory 17463 with a size of 1.07 GB.
That's not a subdirectory, but a datafile segment.

> But there are also 17 subdirectories with names from 17463.1 to
> 17.463.17.  There are also other entries with similar forms of
> duplication and suspiciously identical file sizes of 1.07GB.
>

Again, those are files, not subdirectories. Large datafiles are split
into 1GB segments, so for example 10GB table with relfilenode 17463 will
be stored in files 17463, 17463.1, 17463.2, ..., 17463.9

> Is this normal behavior?  Where in the postgres documentation do I
> read up on this? Postgres strikes me as superior to MySQl, especially
> with regard to string functions and regular expressions, but it’s
> harder to look under the hood.

https://www.postgresql.org/docs/current/static/storage-file-layout.html

> How, for instance, do I figure out what number corresponds to the
> table that I know as ‘earlyprinttuples
>

SELECT relfilenode FROM pg_class WHERE relname = 'earlyprinttuples';


regards

--
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--

Melvin Davidson
I reserve the right to fantasize.  Whether or not you
wish to share my fantasy is entirely up to you.

SELECT c.oid,
       n.nspname as schema,
       c.relname as table, 
       (SELECT oid FROM pg_database WHERE datname = current_database() ) as db_dir,
       c.relfilenode as filename
  FROM pg_class c
  JOIN pg_namespace n ON (n.oid = c.relnamespace)
WHERE relname NOT LIKE 'pg_%' AND
      relname NOT LIKE 'information%' AND
      relname NOT LIKE 'sql_%' AND
      relkind = 'r'
ORDER BY 2, relname;


SELECT c.oid, 
       n.nspname as schema,
       c.relname as table, 
       pg_stat_get_last_vacuum_time(c.oid) as last_vacuum,
       pg_stat_get_tuples_inserted(c.oid) as inserted,
       pg_stat_get_tuples_updated(c.oid) as updated,
       pg_stat_get_tuples_deleted(c.oid) as deleted
  FROM pg_class c
  JOIN pg_namespace n ON (n.oid = c.relnamespace)
 WHERE nspname NOT IN ('information_schema', 'pg_toast', 'pg_catalog')
   and relkind = 'r'
 ORDER BY 2, 3;

SELECT db.oid, 
       db.datname,
       au.rolname as datdba,
       pg_encoding_to_char(db.encoding) as encoding,
       db.datallowconn,
       db.datconnlimit,
       db.datfrozenxid,
       tb.spcname as tblspc,
--       db.datconfig,
       db.datacl
  FROM pg_database db
  JOIN pg_authid au ON au.oid = db.datdba
  JOIN pg_tablespace tb ON tb.oid = db.dattablespace
 ORDER BY 2; 

SELECT oid, 
       datname, 
       pg_size_pretty(pg_database_size(datname))as size_pretty, 
       pg_database_size(datname) as size,
       (SELECT pg_size_pretty (SUM( pg_database_size(datname))::bigint)  
          FROM pg_database)  AS total,
       ((pg_database_size(datname) / (SELECT SUM( pg_database_size(datname))  
                                       FROM pg_database) ) * 100)::numeric(6,3) AS pct
  FROM pg_database 
  ORDER BY datname;