Most likely your data set coming back is too large to process all at once. You need to set the fetch size on the statement to control the number of records coming back per request for them.
ps.setFetchSize( 100 ); where ps is a prepared statement. The result set will use a cursor to only bring back 100 records at a time and will automatically fetch the next hundred until the entire result set has been brought back.
Make sure that your JDBC statements don't have a semi colon at the end of them otherwise the setFetchSize will be ignored and the entire result set will be returned at once.
--sean
Rachel McConnell wrote:
Hello,
I have a Java web application using Hibernate to connect to a PostgreSQL backend. I am seeing the below stack trace during processing of a set of data consisting of around 1000 objects; for a 200 object set I don't see the exception.
I believe the salient point is the Out Of Memory bit - marked below by *****'s
The fact that this exception occurs during a call to org.postgresql.util.PSQLException.parseServerError() seems disturbing too.
* the JVM or servlet container might be out of memory, however the db is on a different machine than the server so this doesn't seem likely
* Postgres might be out of memory. How would I check for this? Subsequent requests go through just fine, as do any requests I make directly using psql.
* something Else is happening
Any advice on how to attack this issue would be much appreciated!
Rachel McConnell
WARN [ SystemErrorInterceptor ] - java.util.NoSuchElementException
java.util.NoSuchElementException
at java.util.AbstractList$Itr.next(AbstractList.java:426)
at java.util.Collections$6.nextElement(Collections.java:2479)
at com.enginegreen.denali.interceptor.SystemErrorInterceptor.intercept(SystemErrorInterceptor.java:89)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:62)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116)
at com.opensymphony.webwork.dispatcher.ServletDispatcher.serviceAction(ServletDispatcher.java:182)
at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:162)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
at com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96)
at com.enginegreen.denali.chrome.filter.ChromeFilter.doFilter(ChromeFilter.java:57)
at com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
at com.enginegreen.denali.interceptor.HibernateFilter.doFilter(HibernateFilter.java:42)
at com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
at com.enginegreen.denali.action.DenaliFilter.doFilter(DenaliFilter.java:28)
at com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
at com.caucho.server.http.Invocation.service(Invocation.java:315)
at com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135)
at com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346)
at com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274)
at com.caucho.server.TcpConnection.run(TcpConnection.java:139)
at java.lang.Thread.run(Thread.java:534)
WARN [ SystemErrorInterceptor ] - com.enginegreen.denali.persistence.PersistenceException: net.sf.hibernate.JDBCException: could not initialize collection: [com.enginegreen.ledger.model.InvoiceSummary.orderSummarySet#78828]
at com.enginegreen.ledger.invoice.impl.BillingClerkImpl.renderInvoicesAsPDF(BillingClerkImpl.java:429)
at com.enginegreen.denali.domain.billing.BillingManagerImpl.finalizeNextBillingPeriod(BillingManagerImpl.java:248)
at com.enginegreen.denali.domain.billing.SecureBillingManager.finalizeNextBillingPeriod(SecureBillingManager.java:96)
at com.enginegreen.denali.action.billing.GenerateInvoicesAction.doExecute(GenerateInvoicesAction.java:121)
at com.enginegreen.denali.action.WorkflowActionSupport.execute(WorkflowActionSupport.java:65)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:172)
at com.enginegreen.denali.interceptor.ActionSourceInterceptor.intercept(ActionSourceInterceptor.java:50)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.enginegreen.denali.interceptor.NullInterceptor.intercept(NullInterceptor.java:25)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.enginegreen.denali.interceptor.security.AdminSecurityInterceptor.intercept(AdminSecurityInterceptor.java:31)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.enginegreen.denali.interceptor.SystemErrorInterceptor.intercept(SystemErrorInterceptor.java:47)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:62)
at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169)
at com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116)
at com.opensymphony.webwork.dispatcher.ServletDispatcher.serviceAction(ServletDispatcher.java:182)
at com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:162)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
at com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96)
at com.enginegreen.denali.chrome.filter.ChromeFilter.doFilter(ChromeFilter.java:57)
at com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
at com.enginegreen.denali.interceptor.HibernateFilter.doFilter(HibernateFilter.java:42)
at com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
at com.enginegreen.denali.action.DenaliFilter.doFilter(DenaliFilter.java:28)
at com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88)
at com.caucho.server.http.Invocation.service(Invocation.java:315)
at com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135)
at com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346)
at com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274)
at com.caucho.server.TcpConnection.run(TcpConnection.java:139)
at java.lang.Thread.run(Thread.java:534)
Caused by: net.sf.hibernate.JDBCException: could not initialize collection: [com.enginegreen.ledger.model.InvoiceSummary.orderSummarySet#78828]
at net.sf.hibernate.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:287)
at net.sf.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:3226)
at net.sf.hibernate.collection.PersistentCollection.forceInitialization(PersistentCollection.java:340)
at net.sf.hibernate.impl.SessionImpl.initializeNonLazyCollections(SessionImpl.java:3089)
at net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:138)
at net.sf.hibernate.loader.Loader.doList(Loader.java:950)
at net.sf.hibernate.loader.Loader.list(Loader.java:941)
at net.sf.hibernate.hql.QueryTranslator.list(QueryTranslator.java:834)
at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1512)
at net.sf.hibernate.impl.QueryImpl.list(QueryImpl.java:39)
at com.enginegreen.ledger.invoice.impl.BillingClerkImpl.renderInvoicesAsPDF(BillingClerkImpl.java:414)
... 42 more
******
Caused by: org.postgresql.util.PSQLException: ERROR: out of memory
at org.postgresql.util.PSQLException.parseServerError(PSQLException.java:139)
at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:154)
at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:101)
at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)
at org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Statement.java:515)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:50)
at org.postgresql.jdbc1.AbstractJdbc1Statement.executeQuery(AbstractJdbc1Statement.java:231)
at net.sf.hibernate.impl.BatcherImpl.getResultSet(BatcherImpl.java:87)
at net.sf.hibernate.loader.Loader.getResultSet(Loader.java:795)
at net.sf.hibernate.loader.Loader.doQuery(Loader.java:189)
at net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:133)
at net.sf.hibernate.loader.Loader.loadCollection(Loader.java:910)
at net.sf.hibernate.loader.Loader.loadCollection(Loader.java:885)
at net.sf.hibernate.loader.OneToManyLoader.initialize(OneToManyLoader.java:80)
at net.sf.hibernate.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:284)
... 52 more
---------------------------(end of broadcast)--------------------------- TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
