How to set storage for OX Drive to another Server ?

[lw3234]

Hello,

I am currently evaluating the possibilities of open Xchange.

Due to security and performance considerations I want to use two servers for my open Xchange setup.
One providing the frontend webinterface mailserver etc. in a DMZ and another server as backend where the mailboxes and the OX Drive files are stored.
While searching the web for this topics I did not find any proper instruction how to set up such an environment. Iam using univention corporate server (UCS 4.0) on both servers and have no idea where to start to split these functions since everything is after installing open xchange through the appcenter in one place.

Any ideas ?

Thx,
Ludwig

[Wolfgang Rosenauer]

I'm not sure how to implement this with UCS and the preconfigured OX (and if it's even possible).

Usually you would have the webserver component of OX in a DMZ but keep the Java application server in your private network.
To separate webserver and application server you basically just need to change the http_proxy.conf to point to an external machine and make sure that the application server listens not only on the loopback interface.

[lw3234]

Usually you would have the webserver component of OX in a DMZ but keep the Java application server in your private network.

That's exactly what I am up to. In my opinion, as UCS claims to be a Linux platform for corporate business it should offer such an option somehow. I cannot imagine that all their customers just set up an OX environment keeping everything in one place/server.

To separate webserver and application server you basically just need to change the http_proxy.conf to point to an external machine and make sure that the application server listens not only on the loopback interface.

In this case I could image a quick an dirty solution like:
Install 2 servers with UCS and complete OX, then changing the http_proxy.conf in the dedicated frontend pointing to the other which then will become the backend. Afterwards one could uninstall the unecessary components on the frontend left from the "complete setup".

[Wolfgang Rosenauer]

I really do not know enough about UCS administration. My guess would be to install OX on the "internal" server with full integration while on the outer one you just install Apache and one or two open-xchange packages required on the webserver manually and manually configure the Apache.

Probably someone else here has more experience with UCS and OX for that purpose or otherwise you could try Univention forums.

[lw3234]

After posting my topic in the UCS forum I just landed straight back here :-)
http://forum.univention.de/viewtopic.php?f=67&t=3681

UCS does support only a all-in-one installation. For any distributed structure I was redirected to the open xchange documentation.

So... is there any guide etc. available showing how the OX Frontend can be set up on a seperate server using an already existing backend (in my case this would be the complete all-in-one setup in UCS) ?

[lw3234]

After posting my topic in the UCS forum I just landed straight back here :-)
http://forum.univention.de/viewtopic.php?f=67&t=3681

UCS does support only a all-in-one installation. For any distributed structure I was redirected to the open xchange documentation.

So... is there any guide etc. available showing how the OX Frontend can be set up on a seperate server using an already existing backend (in my case this would be the complete all-in-one setup in UCS) ?

[lw3234]

Just found a short description on how to set up OX in a distributed UCS environment while digging deep in the documentation on the OX website:

http://software.open-xchange.com/OX6...XSE4UCS_en.pdf

I will start from there, setting up a backend with the Mysql database an OX instance on one machine and on another IMAP, spam-/virus filter etc. as a frontend.

[lw3234]

Hello,

I just want to report my experiences about the setup described in the article ("Installation in a distributed environment") mentioned above:

Following precisely the description I set up:
1x UCS DC Master
1x UCS DC Backup (Backend)
1x UCS DC Slave (Frontend)

On the backend I installed the mysql DB and the active OX instance. (Packets: mysql-server and univention-ox)
On the frontend I intalled Imap and Smtp as described (Packet: univention-mail-cyrus-ox)

My exspectation from that was to end up with on distributed OX installation where the internal OX server is using the frontend-servers Imap and Smtp features to communicate.

What I got was:
- an fully installed (incl. Imap, Smtp etc.) OX instance on the backend
- an fully installed cyrus-ox instance on the frontend

where the backend was using its own (local) imap and smtp server leaving the frontend completely out of concern.

Thank you "tutorial" this was more than wasting of time since I just could have installed OX through the UCS app-center with exactly the same result having 1 fully configured OX server without any distribution of features at all.


At this point I want to lose some words about the online availabe OX documentation:

First of all, I am not a lazy person and you can trust me, when I am saying that I was searching the OX Wiki and all documentation up and down to found information concerning my issues. However, I (except the "guide" mentioned above) found nothing that truely helped me. Even when searching for basic information like "where are the ox-instances files are stored under my linux filesystem" I found nothing. When searching for "best practises" on how to set up OX... I found nothing.

I understand that open xchange is no charity foundation and offers the community-edition as it is without support. That's fine. But it is no reason (!my assumption!) to hold back general information and documentation for paying customers only, as in my case Iam considering to by a future paying customer but want to evaluate the possiblities before. Furthermore, waiting for 3 days until my post underwent some moderation does not help as well.

Finally here the quick answer to the topics title "How to set storage for OX Drive to another Server":
The files for oxdrive/infstore are located on the OX-Server under "/var/oxfilestore" these folder can be easily moved to another server and afterwards be mounted throught NFS back on the OX-Server. Done.

Since this topic now is finished, I will open a new one regarding best practises for distributed OX setup :-)

[Wolfgang Rosenauer]

At this point I want to lose some words about the online availabe OX documentation:

First of all, I am not a lazy person and you can trust me, when I am saying that I was searching the OX Wiki and all documentation up and down to found information concerning my issues. However, I (except the "guide" mentioned above) found nothing that truely helped me. Even when searching for basic information like "where are the ox-instances files are stored under my linux filesystem" I found nothing. When searching for "best practises" on how to set up OX... I found nothing.

I understand that open xchange is no charity foundation and offers the community-edition as it is without support. That's fine. But it is no reason (!my assumption!) to hold back general information and documentation for paying customers only, as in my case Iam considering to by a future paying customer but want to evaluate the possiblities before. Furthermore, waiting for 3 days until my post underwent some moderation does not help as well.

Finally here the quick answer to the topics title "How to set storage for OX Drive to another Server":
The files for oxdrive/infstore are located on the OX-Server under "/var/oxfilestore" these folder can be easily moved to another server and afterwards be mounted throught NFS back on the OX-Server. Done.

Since this topic now is finished, I will open a new one regarding best practises for distributed OX setup :-)

While I understand your concerns about the documentation in the wiki please let me tell you that the documentation is all the same for everyone. Paying and not paying and also mainly for internal staff. If you work all day with Open-Xchange and our wiki its structure is much clearer. Therefore sometimes it's very hard for us what kind of documentation is missing or unclear.
Sometimes it's also hard to find the relevant information since we also have old documentation in the wiki which might lead to misunderstandings.
BTW, I hope you found those tutorials: (like this as the smallest one)

http://oxpedia.org/wiki/index.php?title=AppSuite:OX_Tutorial_10K

[lw3234]

Thank you for your explanation. I agree with that and I also found out that most things are described in one way or another.
However, from the side it is quite difficult to get what you need. For example the information about the location of the infostore I found while searching for information how to backup an OX setup. This was the only location where I found anything about the filestructure of OX.

BTW, I hope you found those tutorials: (like this as the smallest one)
http://oxpedia.org/wiki/index.php?ti...X_Tutorial_10K

That one I found, but it is scratching only the surface of everything and when it comes to UCS everything ends with "go to the App Center and click install"

Hence, my main problem is not so much the documentation of OX in detail (which seems to be ok for standard linux distributions after you got used to it) but the combination of OX + UCS.
I remember you advised me to ask about UCS concerns in the UCS forum. Guess what... they told me exactly the same, another way round.
UCS says: "OX is not our product and for some reason, about how it is implemented in our environment, we have no idea"
OX says: "UCS is not our product and even though it is strongly implemented into that environment we have no idea how"


Is it so exotic to use OX on UCS ?

Before you are asking: Why I do not just install a regular linux distribution and use OX directly ? I answer this in upfront.
Cause I have already an existing UCS domain providing ldap, active directory, samba and several other services. I like that everything can be managed through the webinterface in a central manner. Therefore I now want to integrate OX into that environment with the benefits of central user administration and configuration etc.
Here the current solution offered for OX is a all-in-one solution on a "take-it-or-leave-it" basis. Installing everything in one place without any freedom of individual configuration.

Where Iam now on my way of evaluation?

I started all over again and used the before mentioned guideline for installing OX in an distributed environment under UCS another time.

1. Setup the backend server (UCS DC Backup) installing the mysql database and providing an NFS share for the filestore
2. Setup the front server (UCS DC Slave) installing the OX instance including cyrus and postfix since it is part of the packages' dependencies ("univention-ox")
3. Moved the filestore to the NFS share from the frontend to the backend.

As I want all data stored on the backend using the frontend really only as application server and gateway for users to connect to I have now only one problem left: Cyrus.
Cyrus does not support to store its imap folders and mails on an external server (technically). Furthermore it is pretty old and absolutely not state of the art at all anymore. Even OX prefers in their documentation dovecot.

4. So... I tried to remove cyrus and postfix from the frontend and found out, that these packages are direct dependencies of the "univention-ox" package which is the main OX application. Hence removing that (in any case not necessary services) would cause uninstalling OX at all. Finally I used "dpkg" to manipulate the dependencies of the package "univention-ox" making it possible for me to remove successfully cyrus and postfix from the frontend and keeping the OX untouched.

5. OX still works fine and also UCS webadministration was updated successfully. Now I wanted to install dovecot and exim on the frontend and found out that both are not available in the univention repository. Why? Because UCS does not officially "support" them as only cyrus+postfix does integrate itself in the UCS interface. Therefore I installed them from the debian repository.

(6.) Dovecot in exim are basically working and I am currently dealing with the setup of the ldap authentication with UCS as this is now on me after using non official univention-packages
.
(7.) After that will be done, I will move dovecots mail store to the NFS and viola I (hope to) end up with a very nice OX/Mail frontend and a complete DB+Filestore+MailDB backend managable through UCS.


Question then will be: What will happen if I want to install the OXExtender for Outlook and Activesync support ? I suppose it will crash because of missing dependencies etc etc etc.... (same I asume for any updates of OX in future)

I will see, I do not give up... ;-)

[Wolfgang Rosenauer]

AFAIK the OX4UCS is really mainly designed for small setups where it works out of the box and can easily be set up and managed.
I still would say you can do almost every combination even with UCS but we don't have documentation about it and also Univention does not have it.
I guess also that replacing Cyrus with Dovecot is possible as well but again this has never been tested and prepared to work.
So yes, it is exotic to use a distributed OX on a UCS cluster especially in your interpretation of backend and frontend.
In OX land a frontend is basically just an Apache (and for IMAP probably a nginx oder Dovecot proxy/director). Then there is the middleware (OX application server) and finally a backend (MySQL, filestorage, IMAP server). There are IMHO just too many possible combinations to support them all.

Additional features of OX (like Outlook and EAS support, or e.g. Documents) should only have an impact on the application server and OX packages should have only dependencies on other OX components but not these disconnected ones to cyrus for example. They are a dependency of the univention-ox integration because that one is integrated with Cyrus and not Dovecot. This does not help you much though :-(

[lw3234]

AFAIK the OX4UCS is really mainly designed for small setups where it works out of the box and can easily be set up and managed.

You are right. I just had another picture of it when I started to test this setup.

So yes, it is exotic to use a distributed OX on a UCS cluster especially in your interpretation of backend and frontend.
In OX land a frontend is basically just an Apache (and for IMAP probably a nginx oder Dovecot proxy/director). Then there is the middleware (OX application server) and finally a backend (MySQL, filestorage, IMAP server).

I do not see so much difference between my interpretation and yours. The only thing I did is to not use any middleware but place the OX application server together with the Apache and the IMAP server as frontend. The backend definition is them same.
My main concern in doing so is on the one hand to save resources (hardware) and on the other hand to provide enhanced security of the user data by not offering it directly with one system to anybody on a frontend facing the internet. Or can the OX application server facing the internet be seen as a security risk ?

Additional features of OX (like Outlook and EAS support, or e.g. Documents) should only have an impact on the application server and OX packages should have only dependencies on other OX components but not these disconnected ones to cyrus for example.

This exactly I already have proofen wrong. e.g. when trying to remove the package "cyrus-imapd-2.4" it showed my a direct dependency on "univention-ox" and some more OX related packages telling that they will be removed all together. So the problem I see is that also the additional features of OX under UCS are packed with such dependencies and thus can be not installed without further adaptation. However, this is just guessing and we will see when I come to that point.

Nevertheless, UCS and OX could make things here much easier. It is not necessary to support all the possibilities, but at least it should be supported to be able to use these options. Ok, provide on package like "univention-ox" installing everything all in one, but please consider something like "univention-ox-core" that integrates with UCS but does not install cyrus and postfix leaving the choice open to select something else. (<- I know, that is something to address to UCS)

[meybohm]

Hi,

let me try to shed some light on the points mentioned here.
First of all, as already said, the OX App is designed as a simple-to-install, out-of-the-box solution and therefore lacks some flexibility in favour of simplicity. Because of this you will need to make some basic changes in integration/installation procedure to create a distributed OX setup together with the UCS integration.

No need to say that switching the complete mail stack from Postfix/Cyrus to Exim/Dovecot is massive and inversive customization added on top of that.
You should also keep in mind that the UMC integration (like adding a new user) is strongly bound to the use of Postfix and Cyrus as the so called "Listener-Modules" use commands like "cyradm" to manage mailboxes etc. Those parts, as well as LDAP lookup for mail addresses, groups, possibly also mail domains and user authentication would need to be re-implemented for a Exim/Dovecot setup.

5. OX still works fine and also UCS webadministration was updated successfully. Now I wanted to install dovecot and exim on the frontend and found out that both are not available in the univention repository. Why? Because UCS does not officially "support" them as only cyrus+postfix does integrate itself in the UCS interface. Therefore I installed them from the debian repository.

Those packages are not part of the "maintained" UCS repository but you can (and probably should) install them from our "unmainained" repository which you may activate via UCR like:

ucr set repository/online/unmaintained=yes

Nevertheless, UCS and OX could make things here much easier. It is not necessary to support all the possibilities, but at least it should be supported to be able to use these options. Ok, provide on package like "univention-ox" installing everything all in one, but please consider something like "univention-ox-core" that integrates with UCS but does not install cyrus and postfix leaving the choice open to select something else. (<- I know, that is something to address to UCS)

As you already mentioned, the "univention-ox" Package ( which is kind of a "meta-package" for installation and configuration of the OX App) has dependencies to OX packages (like "open-xchange-grizzly"), more ucs specific OX meta packages like "open-xchange-meta-oxucs", "univention-ox-common", "univention-ox-framework", the OX-UCS mail stack ("univention-mail-postfix-ox"), MySQL and UMC integration (UMC modules for LDB access, mail-domain, user configuration and stuff like that).
Again most of those dependencies are meta-packages themselves that break down into the actual software packages (like "cyrus-imapd-2.4" being installed as a dependency of "cyrus-common-2.4" which is installed by "univention-mail-cyrus" which is a dependency of "univention-mail-cyrus-ox" and that comes by installing "univention-ox-meta-singleserver" via the App Center).

That means it is basically possible to install most if not all of the components manually without the need of meta-packages like "univention-ox" or "univention-ox-meta-singleserver" but you need to know which packages you need and if/what configuration or development you manually have to do if you don't want to use the meta-packages.
Please see the your post in our forum for that where I just outlined some possible steps on how to achieve kind of frontend/backend setup:
Open Xchange Installation als Front-/Backend Variante


Regards,
Janis Meybohm

[lw3234]

Hello,

I wanted to come back with the latest news from my experiment:

First of all, due to reasons of easier implementation I decided not to use exim instead of postfix and therefore setup the combination postfix + dovecot.
Since postfix is already completely configured after UCS + OX installation (incl. Clamdav and Spamassassin) only dovecot needed to be adopted.
So far this construction works fine without. OX passes the credentials to dovecot which authenticates the user then against ldap and provides IMAP access. In case the user does not have a mailbox yet, dovecot creates the mail folders.

However, one issue/question is left. If I set up a new mail domain in OX, adding it to the same context and creating a new user whos primary mailaddress is named after the new domain no request from OX is sent to dovecot during login. Therefore no mailbox is created etc.... If I set the primary mailaddress of that user to the first domain, just adding the second one as alternative mailaddress everything works as exspected.

Any ideas, why OX (according to the log files) not even tries to contact the imap server in case of a new domain?