PostgreSQL 9.4.0
Are generalizing here or are you really running 2+ year old patch version?
Why pg_class table is getting bigger in size.
Because you are creating (specific) objects.
How to stop increasing it.
Stop creating (those specific) objects.
Does it affect the performance.
It can - depends greatly on scale.
Note, frequent usage of temporary tables is a common cause for this kind of behavior.
David J.