Homepage | Products | OX Knowledge Base | Support | Try Now | Contact | Company
OX Logo
Results 1 to 4 of 4
  1. #1
    skosner Guest

    Default Unable to log into groupware app - variety of error messages

    Hi,

    Have been unable to log into groupware web interface.
    Attempts usually produce the following error - "503 - service temporarily unavailable"
    Though occasionally the error is one of the following instead...

    "500 - internal server error"
    "Error: Cannot get connection to database. (CTX-0005, -618270995-371)"
    "07-08-2010 12:28 AM-->Error: A socket error occurred: Too many open files (MSG-1021, 618471929-2272)"

    Rebooting the server always fixes the problem, but only temporarily. After another day or so, the problem recurs and cannot log in until the server is rebooted again. Accordingly suspected that maybe a service that launches upon boot, is getting messed up or getting shutdown while the server is running. So tried manually starting each service that automatically launches on startup, and a few other things, but nothing helped.
    This is what was attempted:

    1) stopped and started the groupware
    2) Ran all these commands...

    a2enmod proxy
    a2enmod proxy_ajp
    a2enmod expires
    a2enmod deflate
    a2enmod headers

    insserv mysql
    insserv apache2
    insserv open-xchange-groupware
    insserv open-xchange-admin

    checking the log files did not show anything conspicuous, but maybe the wrong ones were checked.

    Searching through the forum returned too many threads complaining about "503 - service temporarily unavailable", so instead of an inexperienced user trying to figure out which thread is relevant, could you point out which one(s) might help with this problem and/or give some guidance?

    Linux OpenSuse 11.2
    OX Version : 6.16.1 Rev3 ( 2010-04-23 11:06:42)

    Thank you
    Last edited by skosner; 07-28-2010 at 05:44 AM.

  2. #2
    Join Date
    Feb 2007
    Location
    Germany
    Posts
    3,695

    Default

    Hi,

    seems like your server runs out of resources. Is it some kind of VPS with limited resources? Are there monitoring information about memory consumption and open file descriptors?

    Greetings
    Last edited by Martin Heiland; 07-28-2010 at 09:15 AM.

  3. #3
    skosner Guest

    Default

    Hi,

    System resources seem to be more than sufficient and OX ran fine for awhile before this problem began. There have been no changes/reduction in system resources at any point.

    System details:
    Dual P3 1ghz processors, 1GB ram with 75% consumed, swap partition is 1.5GB, which is barely used, hard disc is 32GB with 24GB free, set up as a unified file system - 1 partition.

    After further troubleshooting, it seems that AJP is the cause of the issue. When OX is running fine, AJP13 is running also on the open port 8009/TCP.
    But when the problem occurs, AJP13 has stopped running. So the question now is what would be stopping AJP?

    We are NOT running tomcat. Other than the lightly loaded web server, we do not run any other server applications.

    We found some suspicious entries in the logs...

    These errors are scattered throughout the file /var/log/localmessages, but a cluster of them were at the end of the file:

    Code:
    Aug  1 12:20:10 GrandCentral checkproc: checkproc: can not get session id for process 4021!
    Aug  1 13:42:17 GrandCentral checkproc: checkproc: can not get session id for process 6313!
    Aug  1 17:36:23 GrandCentral checkproc: checkproc: can not get session id for process 6003!
    Aug  1 19:37:50 GrandCentral checkproc: checkproc: can not get session id for process 7823!
    Aug  1 20:12:17 GrandCentral checkproc: checkproc: can not get session id for process 10892!
    Aug  1 22:14:40 GrandCentral checkproc: checkproc: can not get session id for process 13279!
    Aug  2 21:51:05 GrandCentral checkproc: checkproc: can not get session id for process 12479!
    Aug  2 23:10:42 GrandCentral checkproc: checkproc: can not get session id for process 16089!
    In open-xchange-console.log:

    Code:
    Could not find fetcher.properties on classpath
    java.net.SocketException: Too many open files
        at java.net.Socket.createImpl(Socket.java:414)
        at java.net.Socket.connect(Socket.java:544)
        at java.net.Socket.connect(Socket.java:495)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:174)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:240)
        at sun.net.www.http.HttpClient.New(HttpClient.java:321)
        at sun.net.www.http.HttpClient.New(HttpClient.java:338)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:814)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:755)
        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:680)
        at com.openexchange.subscribe.crawler.internal.CrawlerUpdateTask.run(CrawlerUpdateTask.java:104)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
        at com.openexchange.threadpool.internal.CustomThreadPoolExecutor$ScheduledFutureTask.runPeriodic(CustomThreadPoolExecutor.java:856)
        at com.openexchange.threadpool.internal.CustomThreadPoolExecutor$ScheduledFutureTask.run(CustomThreadPoolExecutor.java:881)
        at com.openexchange.threadpool.internal.CustomThreadPoolExecutor$Worker.runTask(CustomThreadPoolExecutor.java:737)
        at com.openexchange.threadpool.internal.CustomThreadPoolExecutor$Worker.run(CustomThreadPoolExecutor.java:763)
        at java.lang.Thread.run(Thread.java:636)

    In open-xchange.log.3:

    Code:
    Aug 2, 2010 10:51:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem calling the UDP Discovery Sender [224.0.0.1:6781]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem closing Passive Broadcast sender
    java.lang.NullPointerException
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:171)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:56 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender <init>
    SEVERE: Could not bind to multicast address [224.0.0.1]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:56 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem calling the UDP Discovery Sender [224.0.0.1:6780]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:56 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem closing Passive Broadcast sender
    java.lang.NullPointerException
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:171)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:56 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender <init>
    SEVERE: Could not bind to multicast address [224.0.0.1]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:56 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem calling the UDP Discovery Sender [224.0.0.1:6781]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:51:56 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem closing Passive Broadcast sender
    java.lang.NullPointerException
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:171)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:52:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender <init>
    SEVERE: Could not bind to multicast address [224.0.0.1]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:52:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem calling the UDP Discovery Sender [224.0.0.1:6780]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:52:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem closing Passive Broadcast sender
    java.lang.NullPointerException
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:171)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:52:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender <init>
    SEVERE: Could not bind to multicast address [224.0.0.1]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:52:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem calling the UDP Discovery Sender [224.0.0.1:6781]
    java.net.SocketException: Too many open files
    	at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
    	at java.net.AbstractPlainDatagramSocketImpl.create(AbstractPlainDatagramSocketImpl.java:69)
    	at java.net.DatagramSocket.createImpl(DatagramSocket.java:319)
    	at java.net.DatagramSocket.<init>(DatagramSocket.java:227)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:159)
    	at java.net.MulticastSocket.<init>(MulticastSocket.java:105)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySender.<init>(UDPDiscoverySender.java:67)
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:150)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Aug 2, 2010 10:52:26 AM org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread run
    SEVERE: Problem closing Passive Broadcast sender
    java.lang.NullPointerException
    	at org.apache.jcs.auxiliary.lateral.socket.tcp.discovery.UDPDiscoverySenderThread.run(UDPDiscoverySenderThread.java:171)
    	at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)
    	at java.lang.Thread.run(Thread.java:636)
    Since we are not running in a clustered environment, we changed sessioncache.ccf and cache.ccf based on our impression from the error log files...

    In sessioncache.ccf, we changed several values from true to false:

    19,20c19,20
    < jcs.region.SessionCache.elementattributes.IsRemote =true
    < jcs.region.SessionCache.elementattributes.IsLatera l=true
    ---
    > jcs.region.SessionCache.elementattributes.IsRemote =false
    > jcs.region.SessionCache.elementattributes.IsLatera l=false
    32c32
    < jcs.auxiliary.SessionLTCP.attributes.UdpDiscoveryE nabled=true
    ---
    > jcs.auxiliary.SessionLTCP.attributes.UdpDiscoveryE nabled=false
    34c34
    < jcs.auxiliary.SessionLTCP.attributes.Receive=true
    ---
    > jcs.auxiliary.SessionLTCP.attributes.Receive=false


    In cache.ccf, we also changed some values from true to false.

    16c16
    < jcs.default.elementattributes.IsLateral=true
    ---
    > jcs.default.elementattributes.IsLateral=false
    279c279
    < jcs.auxiliary.LTCP.attributes.Receive=true
    ---
    > jcs.auxiliary.LTCP.attributes.Receive=false
    282c282
    < jcs.auxiliary.LTCP.attributes.IssueRemoveOnPut=tru e
    ---
    > jcs.auxiliary.LTCP.attributes.IssueRemoveOnPut=fal se


    The problem usually recurs within a day, so we should soon have an idea if this worked. However, we're in a rush to settle this issue conclusively, and would like your input regardless. What are your thoughts?

    Thank you

  4. #4
    skosner Guest

    Default

    Hi Martin,

    The problem has recurred. Everything else works well enough, but this one issue makes the product essentially unusable. We're pressed for time to get this working again. Do you have any ideas what might be causing this?

    Thank you

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •