ldap contacts weird desplay in select recipients

[sam]

Hi,

I've got an interesting issue when using ldap contacts in v 6.18. In principle, these work ok if I look into the particular folder under Public folders. But if I open a new email, press the To..." button,
I get very strange display in the "Select recipients" window (see attached picture).

Basically when "Search all contact folders" is checked I get an empty line (just dots in each column) for each ldap contact. Normal Global addressbook entries follows below normally. If I look for a particular ldap contact,
it is found correctly. If I scroll forth and back the "..." entries get popullated by random contacts from Global addressbook.

Could somebody confirm this ? I'm not sure if it is a bug or some configuration issue. I remember seeing this in v6.14 but cannot remember it on 6.16.

[Dennis Sieben]

Hi,

looks like there are colliding ids. Is it possible for you to attach the requests made by the GUI? Can be either done by using FireBug or via a tcpdump of the traffic if the connection isn't secured.

Regards,

Dennis

[sam]

Hello Dennis,

Thanks for reply. I'll try to prepare a wireshark traffic log of the session tomorrow.

Actually I have couple more troubles with ldap contacts at the moment, especially the way distribution lists work (I cannot get them use the list mail address as defined in ldap and used by postfix, ox6 always sends email to all users in distribution list individually, very annoying...). Also, multivalued attributes in ldap doesn't seem to be supported in ldap-contacts.

Regards,

Sam

[Dennis Sieben]

Hello Sam,

Hello Dennis,

Thanks for reply. I'll try to prepare a wireshark traffic log of the session tomorrow.

Actually I have couple more troubles with ldap contacts at the moment, especially the way distribution lists work (I cannot get them use the list mail address as defined in ldap and used by postfix, ox6 always sends email to all users in distribution list individually, very annoying...).

That's nothing special related to contacts-ldap, it's simply the general OX way. If you create distribution lists in OX it works the same way.
What you can do is the following: You could create a separate folder and adjust the contact query, so that the contact query finds the distribution list. This way you won't see the members in OX but you can sent a mail to the distribution list. Note that the lists will then be displayed as contacts

Also, multivalued attributes in ldap doesn't seem to be supported in ldap-contacts.

Could you please tell me in detail which multivalued attributes don't work. In some cases this had be implemented. Maybe some important parts were left out.

Regards,

Dennis

[sam]

Hello Dennis,

sorry, it took me a bit longer to get time to have a look at it. Looking at the gunzipped fragment of the ox server reply from wireshark, it seems clear that there is an ID collision.

This is what I observe:

- the reply is a long tuple which contains all expected entries
- it begins with ldap contacts, then global users and then private contacts follow
- every entry has form [ id, displayName, email, null, null, uid, null, null, givenName, sn ]
- email & Displayname are ok
- uid is the user id from mysql for system users or null for ldap contacts
- id should be probably the unique id used within the browser session

However, the first 12 entries from ldap have ids 1..12 and this clearly overlaps with the ids of global contacts. I would say, that is a problem at server side. Where should I look further ?

That's nothing special related to contacts-ldap, it's simply the general OX way. If you create distribution lists in OX it works the same way.

Well, thats extremely bad and actually a topic for its own thread... What I like on OX is, it does not try to implement all stuff like mta, lda, message store or whatever else - which try to do some other products (and fail badly on this...). So it is exactly what I want if I use own postfix/dovecot/openldap setup and add OX as an independend "addon" for calendars and have a cool web-ui. The trouble is, in this setup, I have and want to use distribution lists as defined in ldap/postfix. Because only postfix can deliver external emails on distlists, it can do some extra post-processing as well and this is also the delivery way if using other imap clients like thunderbird. If I have a distlist "list1@xyz" with users "user1@xyz" and "user2@xyz" the destination address is always "list1@xyz". What OX does is it decodes list members itself (why ??? WTF?) and sends the mail to "user1" and "user2". No way for repicient to tell it was sent to "list1" or for postfix to do some special stuff.

What is deadly missing is an extra field for distribution list email address which might be optional. If such field is defined, the mails should go to this address and not to each member email separately. OX is not the MTA nor LDA.

What you can do is the following: You could create a separate folder and adjust the contact query, so that the contact query finds the distribution list. This way you won't see the members in OX but you can sent a mail to the distribution list. Note that the lists will then be displayed as contacts

That what did already I guess - I simply configured the distribution lists as normal user contacts. It is kind of "least bad" solution". And as you write, there is of course the wrong icon displayed and I cannot display members due to non-working multivalued attributes (see below) - so I will have to patch the OX sources a bit.
But I suspect, it would be cleaner and easier to patch distribution list with addition list email address (see above).

Could you please tell me in detail which multivalued attributes don't work. In some cases this had be implemented. Maybe some important parts were left out.

Ehm, if I try to map any multivalued attribute to any of the ldap-contact attributes,
only the first value will be shown. But I need to display important stuff like memberId, or email aliases of the contact entry. No way at the moment. The very basic implementation would be to convert the value of a multivalued attribute to a string with comma separated list of values by default. Thats really easy and does not break anything.


Regards,

Sam

[Dennis Sieben]

Hello Sam,

This is what I observe:

- the reply is a long tuple which contains all expected entries
- it begins with ldap contacts, then global users and then private contacts follow
- every entry has form [ id, displayName, email, null, null, uid, null, null, givenName, sn ]
- email & Displayname are ok
- uid is the user id from mysql for system users or null for ldap contacts
- id should be probably the unique id used within the browser session

However, the first 12 entries from ldap have ids 1..12 and this clearly overlaps with the ids of global contacts. I would say, that is a problem at server side. Where should I look further ?

This is nothing you can fix. The GUI has to request the folder id now, as the object id itself cannot be guaranteed to be unique over all folders any more. I've opened up a bug here on side for this issue.

Well, thats extremely bad and actually a topic for its own thread... What I like on OX is, it does not try to implement all stuff like mta, lda, message store or whatever else - which try to do some other products (and fail badly on this...). So it is exactly what I want if I use own postfix/dovecot/openldap setup and add OX as an independend "addon" for calendars and have a cool web-ui. The trouble is, in this setup, I have and want to use distribution lists as defined in ldap/postfix. Because only postfix can deliver external emails on distlists, it can do some extra post-processing as well and this is also the delivery way if using other imap clients like thunderbird. If I have a distlist "list1@xyz" with users "user1@xyz" and "user2@xyz" the destination address is always "list1@xyz". What OX does is it decodes list members itself (why ??? WTF?) and sends the mail to "user1" and "user2". No way for repicient to tell it was sent to "list1" or for postfix to do some special stuff.

What is deadly missing is an extra field for distribution list email address which might be optional. If such field is defined, the mails should go to this address and not to each member email separately. OX is not the MTA nor LDA.

We already have a user story for this. This is all covered by a big change for the complete group/distribution list handling. But I can't tell when this will be done. For now the workaround is unfortunately the only solution we can provide.

Ehm, if I try to map any multivalued attribute to any of the ldap-contact attributes,
only the first value will be shown. But I need to display important stuff like memberId, or email aliases of the contact entry. No way at the moment. The very basic implementation would be to convert the value of a multivalued attribute to a string with comma separated list of values by default. Thats really easy and does not break anything.

You're right that's easy to change, and IIRC this was exactly the way it worked in one of the first implementations. But in some environments this will break things as this way won't work for all attribute types. And you need to differentiate on a per attribute base as you can't break it down to the attribute type in general, this will blow up the mapping file for a small benefit.

If you want to change it on your system, you just need to adjust com.openexchange.contacts.ldap.ldap.LdapGetterJNDI Impl. There are different types of getter methods depending on the type of information expected. There's also a distinction between multi-value or not, and there you could adjust it to you special needs.

Regards,

Dennis

[sam]

Hello Dennis,

its me again

Hello Sam,
This is nothing you can fix. The GUI has to request the folder id now, as the object id itself cannot be guaranteed to be unique over all folders any more. I've opened up a bug here on side for this issue.
Dennis

Wow, so nobody noticed this apparent issue so far ? Could you please send give me a hint when the related fix gets available in the development snapshots ?

Hello Sam,
We already have a user story for this. This is all covered by a big change for the complete group/distribution list handling. But I can't tell when this will be done. For now the workaround is unfortunately the only solution we can provide.
Dennis

Wow again ! I was afraid, the current status is considered a must-have / won't fix philosophy/feature of ox. Actually there was already something about groups on old roadmap scheduled around 6.16, so I was scared the current situation is already the improved one. Do you have at least an idea if this change will happen in term of months or years ?

Speaking of groups, there is currently another strange limitation: setting permissions for the ACL-groups do not work since the groups are greyed-out in selection dialogue when setting permissions . Modern IMAP-clients (e.g. dovecot >=1.2) support fully ACL groups and take care of group membership. So in order to set ACL group permission, the client has to do the same as for user, the only extra is to prepend a $-sign in front of the group name. The other way works ok, if I set group-ACL from withon other client (e.g. Thunderbird), it is interpreted correctly by OX. Could you give me a hint, where in code to enable/ungrey groups and add the $-sign?

If you want to change it on your system, you just need to adjust com.openexchange.contacts.ldap.ldap.LdapGetterJNDI Impl. There are different types of getter methods depending on the type of information expected. There's also a distinction between multi-value or not, and there you could adjust it to you special needs.
Dennis

I cannot imagine, why there could have been issues when enabling this by default, but
der Teufel schläft ja nie ...
I go for this patch for now, but its ugly. This should actually be configurable in config file, multivalue-attributes are pretty common in LDAP, to deliberately use just the first one by default is not a good idea.

This brings me to the last question: how to contribute changes back to the project ? OX seems to be a kind of one-way open-source project. E.g. I changed recently the password-changescript package to interpret the script return values (i.e. ldap error, password too weak, password too short, ...) and throw different exception for these so user gets a meaningfull error message why the operation failed. Now I would like this to be included as standard (and I'm sure many others as well) - but how ?

Thanx a lot for your replies,

Regards,

Sam

[Dennis Sieben]

Hello Sam,

Hello Dennis,

its me again


Wow, so nobody noticed this apparent issue so far ? Could you please send give me a hint when the related fix gets available in the development snapshots ?

This one already got fixed in the meantime. So give it a try...

Wow again ! I was afraid, the current status is considered a must-have / won't fix philosophy/feature of ox. Actually there was already something about groups on old roadmap scheduled around 6.16, so I was scared the current situation is already the improved one. Do you have at least an idea if this change will happen in term of months or years ?

Unfortunately not :-(

Speaking of groups, there is currently another strange limitation: setting permissions for the ACL-groups do not work since the groups are greyed-out in selection dialogue when setting permissions . Modern IMAP-clients (e.g. dovecot >=1.2) support fully ACL groups and take care of group membership. So in order to set ACL group permission, the client has to do the same as for user, the only extra is to prepend a $-sign in front of the group name. The other way works ok, if I set group-ACL from withon other client (e.g. Thunderbird), it is interpreted correctly by OX. Could you give me a hint, where in code to enable/ungrey groups and add the $-sign?

Hmm. You should ask this question in another thread, with an IMAP related topic, personally I don't know about the IMAP implementation in detail. But if you open another one. The corresponding developers may see and thread and make some comments.

I cannot imagine, why there could have been issues when enabling this by default, but
der Teufel schläft ja nie ...
I go for this patch for now, but its ugly. This should actually be configurable in config file, multivalue-attributes are pretty common in LDAP, to deliberately use just the first one by default is not a good idea.

The config file is already overloaded and confusing, so I won't feel well with the idea of adding more stuff to it. But I will keep that item on my list, if there are more comments for multivalue handling we might change that in the future, but for now this is the first one. And the one-in-a-million thing currently isn't worth handling it ;-).

This brings me to the last question: how to contribute changes back to the project ? OX seems to be a kind of one-way open-source project. E.g. I changed recently the password-changescript package to interpret the script return values (i.e. ldap error, password too weak, password too short, ...) and throw different exception for these so user gets a meaningfull error message why the operation failed. Now I would like this to be included as standard (and I'm sure many others as well) - but how ?

Yes we don't allow to modify the sources externally at the moment. It might be possible that we allow some dedicated people to contribute changes to our version control systems. But in general it's read-only.

The first way to get your patches into our code right now is creating an enhancement bug. Then a developer will look over the patches and include them if the code quality corresponds to our standards. And please mention the bug numbers here so I can indicate that to the developers if they don't saw it.

Regards,

Dennis