OX Crash when accessing big Email-attachments

[hubsif]

Hi.

I just installed OX 6.6 on Redhat, and I'm having the following problem:

When accessing an Email with a large attachment, OX hangs and then does not display the mail correctly. The log shows error messages like the following:

May 20, 2009 10:21:01 AM com.openexchange.ajp13.AJPv13Watcher$Task run
INFO: AJP Listener "AJPListener-00003" exceeds max. running time of 60000msec -> Processing time: 180639msec
java.lang.Throwable
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream. java:129)
at com.sun.mail.util.TraceInputStream.read(TraceInput Stream.java:110)
at java.io.BufferedInputStream.read1(BufferedInputStr eam.java:254)
at java.io.BufferedInputStream.read(BufferedInputStre am.java:313)
at com.sun.mail.iap.ResponseInputStream.readResponse( ResponseInputStream.java:156)
at com.sun.mail.iap.Response.<init>(Response.java:96)
at com.sun.mail.imap.protocol.IMAPResponse.<init>(IMA PResponse.java:61)
at com.sun.mail.imap.protocol.IMAPResponse.readRespon se(IMAPResponse.java:135)
at com.sun.mail.imap.protocol.IMAPProtocol.readRespon se(IMAPProtocol.java:261)
at com.sun.mail.iap.Protocol.command(Protocol.java:31 0)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAP Protocol.java:1312)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAP Protocol.java:1304)
at com.sun.mail.imap.protocol.IMAPProtocol.fetchBody( IMAPProtocol.java:1044)
at com.sun.mail.imap.protocol.IMAPProtocol.fetchBody( IMAPProtocol.java:1033)
at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage. java:674)
at com.openexchange.mail.mime.dataobjects.MIMEMailPar t.createMultipart(MIMEMailPart.java:300)
at com.openexchange.mail.mime.dataobjects.MIMEMailPar t.getEnclosedCount(MIMEMailPart.java:311)
at com.openexchange.mail.mime.dataobjects.MIMEMailMes sage.getEnclosedCount(MIMEMailMessage.java:155)
at com.openexchange.mail.parser.MailMessageParser.par seMailContent(MailMessageParser.java:280)
at com.openexchange.mail.parser.MailMessageParser.par seMailMessage(MailMessageParser.java:165)
at com.openexchange.mail.parser.MailMessageParser.par seMailMessage(MailMessageParser.java:144)
at com.openexchange.mail.json.writer.MessageWriter.wr iteMailMessage(MessageWriter.java:126)
at com.openexchange.ajax.Mail.actionGetMessage(Mail.j ava:972)
at com.openexchange.ajax.Mail.actionGetMessage(Mail.j ava:817)
at com.openexchange.ajax.request.MailRequest.action(M ailRequest.java:156)
at com.openexchange.ajax.Multiple.doAction(Multiple.j ava:431)
at com.openexchange.ajax.Multiple.parseActionElement( Multiple.java:184)
at com.openexchange.ajax.Multiple.doPut(Multiple.java :130)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:619)
at com.openexchange.ajax.AJAXServlet.service(AJAXServ let.java:380)
at com.openexchange.ajax.SessionServlet.service(Sessi onServlet.java:152)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:689)
at com.openexchange.ajp13.AJPv13RequestHandler.doServ letService(AJPv13RequestHandler.java:575)
at com.openexchange.ajp13.AJPv13Request.response(AJPv 13Request.java:108)
at com.openexchange.ajp13.AJPv13RequestHandler.create Response(AJPv13RequestHandler.java:385)
at com.openexchange.ajp13.AJPv13Connection.createResp onse(AJPv13Connection.java:192)
at com.openexchange.ajp13.AJPv13Listener.run(AJPv13Li stener.java:245)
at java.lang.Thread.run(Thread.java:595)
May 20, 2009 10:23:23 AM com.openexchange.ajp13.AJPv13Watcher$Task run
INFO: AJP Listener "AJPListener-00003" exceeds max. running time of 60000msec -> Processing time: 322160msec
java.lang.Throwable
at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage. java:696)
at com.openexchange.mail.mime.dataobjects.MIMEMailPar t.createMultipart(MIMEMailPart.java:300)
at com.openexchange.mail.mime.dataobjects.MIMEMailPar t.getEnclosedCount(MIMEMailPart.java:311)
at com.openexchange.mail.mime.dataobjects.MIMEMailMes sage.getEnclosedCount(MIMEMailMessage.java:155)
at com.openexchange.mail.parser.MailMessageParser.par seMailContent(MailMessageParser.java:280)
at com.openexchange.mail.parser.MailMessageParser.par seMailMessage(MailMessageParser.java:165)
at com.openexchange.mail.parser.MailMessageParser.par seMailMessage(MailMessageParser.java:144)
at com.openexchange.mail.json.writer.MessageWriter.wr iteMailMessage(MessageWriter.java:126)
at com.openexchange.ajax.Mail.actionGetMessage(Mail.j ava:972)
at com.openexchange.ajax.Mail.actionGetMessage(Mail.j ava:817)
at com.openexchange.ajax.request.MailRequest.action(M ailRequest.java:156)
at com.openexchange.ajax.Multiple.doAction(Multiple.j ava:431)
at com.openexchange.ajax.Multiple.parseActionElement( Multiple.java:184)
at com.openexchange.ajax.Multiple.doPut(Multiple.java :130)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:619)
at com.openexchange.ajax.AJAXServlet.service(AJAXServ let.java:380)
at com.openexchange.ajax.SessionServlet.service(Sessi onServlet.java:152)
at javax.servlet.http.HttpServlet.service(HttpServlet .java:689)
at com.openexchange.ajp13.AJPv13RequestHandler.doServ letService(AJPv13RequestHandler.java:575)
at com.openexchange.ajp13.AJPv13Request.response(AJPv 13Request.java:108)
at com.openexchange.ajp13.AJPv13RequestHandler.create Response(AJPv13RequestHandler.java:385)
at com.openexchange.ajp13.AJPv13Connection.createResp onse(AJPv13Connection.java:192)
at com.openexchange.ajp13.AJPv13Listener.run(AJPv13Li stener.java:245)
at java.lang.Thread.run(Thread.java:595)

When accessing the IMAP-Server with Thunderbird everything is fine and fast.

And I found a setting in mail.properties: com.openexchange.mail.attachmentDisplaySizeLimit=8 192

App-System: RHEL 5.3
java-1.5.0-sun-1.5.0.17-1jpp.2.el5
OX-Version: 6.8.0-2.1

GUI-System: RHEL 5.3
OX-GUI-Version: 6.8.0-4.1
cyrus-imapd-2.3.7-2.el5

Thanks for any help!
hubsif.

[Martin Heiland]

Hi,

what attachment size are we talking about?

Thanks!

[hubsif]

sorry, I forgot: (only) 30MB.

[Carsten Hoeger]

According to the logs and as far as I understand them, OX did not succeed for 3 minutes to write the data to the imap server.

So it might be a problem with the imap server.

[Martin Heiland]

Confirmed, however - this is not an application crash but the timeout for fetching data from the IMAP Server is reached and the connection is closed. As a consequence the E-Mail is not shown properly. While Thunderbird and other clients provide a local data storage for E-Mail they're usually faster than a webmail solution when it comes to file handling etc. There are some steps we could do now:

1. Check why fetching 30MB from the IMAP server takes more than a minute (your turn)
2. Handle E-Mail attachments smarter, e.g. don't download them when opening a mail but just streaming them when actually downloading a attachment. (our turn)

@Carsten: thats correct but we should also ask ourselves why we download the *whole* E-Mail to the OX Server instead of downloading the attachment when it is required.

Thanks a lot for this report.

[hubsif]

Thank you for your reply.

We did some more debugging and located the problem:
The OX-appserver ran out of RAM and did a lot of swapping. We increased the RAM size for the appserver virtual machine and now it takes like 5 seconds to load the 30MB Mail.

Well, sorry for having bothered you. And thank you very much for your quick and qualified answers.

hubsif.

[Martin Heiland]

Hi,

we have dramatically improved the performance and resource usage when opening a E-Mail with large attachments based on your report, thank you very much! Now we display the whole E-Mail instantly, no matter if there is a 10KB or a 500MB attachment and when you decide to download an attachment the data transfer is streamed from the E-Mail backend via the OX Server to your Browser. This way we also solved the memory issue because the whole attachment has been stored to the RAM for caching. Those improvements will be available with the next major release (6.10).

Greetings