From prlw1 at cam.ac.uk  Wed May 14 10:01:37 2008
From: prlw1 at cam.ac.uk (Patrick Welche)
Date: Wed, 14 May 2008 15:01:37 +0100
Subject: autoconf cache variables
Message-ID: <20080514140137.GM5157@quartz.itdept.newn.cam.ac.uk>

Thoughts?

Cheers,

Patrick
(as in, the variables won't get cached because they don't have _cv_ in their
name)
-------------- next part --------------
An embedded message was scrubbed...
From: Patrick Welche <prlw1 at newn.cam.ac.uk>
Subject: autoconf cache variables
Date: Wed, 7 May 2008 16:16:23 +0100
Size: 9893
Url: http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080514/d1c7f1b5/attachment.mht 

From murch at andrew.cmu.edu  Wed May 14 11:16:58 2008
From: murch at andrew.cmu.edu (Ken Murchison)
Date: Wed, 14 May 2008 11:16:58 -0400
Subject: sieve addflag/setflag for \seen flag doesn't work with
	keep/implicit keep
In-Reply-To: <018a01c87908$81df8d70$0b01a8c0@robmhp>
References: <018a01c87908$81df8d70$0b01a8c0@robmhp>
Message-ID: <482B026A.5060600@andrew.cmu.edu>

Sorry for the delay in looking into this.  I looked at the code, and 
finally realized why it was written in the way that it was:

- For fileinto, we use the credentials of the script owner when 
delivering to the mailbox and optionally setting IMAP flags.  We know 
that the script owner has explicitly told us what mailbox to deliver to 
and with which flags.  Of course, the script owner must still have the 
appropriate ACL.

- For keep, we use the credentials of the LMTP authenticated/authorized 
user (if any).  We treat keep no different than a regular LMTP delivery 
where no sieve script is executed.  Yes, an explicit keep is a result of 
a sieve script, but since the destination mailbox was not explicitly 
provided by the script owner, how is the script engine supposed to know 
if the recipient wants some anonymous user to be able to deliver to the 
mailbox and optionally set IMAP flags on messages?  If the ACL is set 
accordingly for a particular auth'd sender (or anonymous), then the 
message can be delivered and possibly have flags set.

If anyone has any thoughts on how to improve on this without creating a 
security hole, I'm all ears.


Rob Mueller wrote:
> The whole mailbox/append code leaves me a bit lost, but I've tracked 
> down the general area of the problem, Ken maybe you can work out what 
> the right fix is, I don't think it's hard.
> 
> sieve_fileinto() does the following:
> 
> 1. cast void * sc -> script_data_t *sd
> 2. pass "sd->username" as the "authuser" param to deliver_mailbox()
> 3. deliver_mailbox() calls append_setup() with "authuser" as the 
> "userid" param
> 4. append_setup() copys the "userid" parameter into the 
> "appendstate.userid" struct area
> 5. during append_commit() "appendstate.userid" is used as the username 
> to add the seen flag to
> 
> That all works fine.
> 
> However, sieve_keep() does the following:
> 
> 1. cast void *mc -> deliver_data_t *mydata
> 2. pass "mydata" as the "mydata" param to deliver_local()
> 3. deliver_local() calls deliver_mailbox(), with "mydata->authuser" as 
> the "userid" param
> 
> The value in mydata->authuser is not the username, in fact they're empty:
> 
> (gdb) p *mydata
> $29 = { ..., authuser = 0x0, authstate = 0x0}
> 
> This means the seen flag never gets stored correctly if you're using 
> keep or the implicit keep, you have to use fileinto.
> 
> I'm not sure of the best way of fixing this. I can see the obvious 
> solution (in sieve_keep, cast void * sc -> script_data_t *sd, and copy 
> authuser and authstate sd to mydata), but that doesn't feel right to me.
> 
> Ken, do you know what the right solution is here?
> 
> Rob
> 
> 


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

From robm at fastmail.fm  Wed May 14 20:52:59 2008
From: robm at fastmail.fm (Rob Mueller)
Date: Thu, 15 May 2008 10:52:59 +1000
Subject: sieve addflag/setflag for \seen flag doesn't work with
	keep/implicit keep
References: <018a01c87908$81df8d70$0b01a8c0@robmhp>
	<482B026A.5060600@andrew.cmu.edu>
Message-ID: <016f01c8b626$074cb820$0a01a8c0@robmhp>


> - For keep, we use the credentials of the LMTP authenticated/authorized
> user (if any).  We treat keep no different than a regular LMTP delivery 
> where no sieve script is executed.  Yes, an explicit keep is a result of a 
> sieve script, but since the destination mailbox was not explicitly 
> provided by the script owner, how is the script engine supposed to know if 
> the recipient wants some anonymous user to be able to deliver to the 
> mailbox and optionally set IMAP flags on messages?  If the ACL is set 
> accordingly for a particular auth'd sender (or anonymous), then the 
> message can be delivered and possibly have flags set.

I don't have a problem with that logic for delivery.

However, the script did explicitly set the \seen flag, regardless of if it 
ends with a keep/fileinto, we should be setting the flag correctly because 
the user explicitly said to.

In fact this does work for every other flag, except the seen flag.

It seems to me the problem is that you're overloading the use of "authuser" 
to mean the "seenuser" as well. Maybe you need an explicit "seenuser" 
parameter that can be passed to deliver_mailbox()? Or if not that, maybe we 
need to get the seenuser based on the actual mailbox name itself that it's 
being delivered to? That seems reasonable, because only a sieve script can 
set the \seen flag, and at the moment there's no way to set the seen flag 
for any user other than the mailbox it's being delivered to?

Rob

>> The whole mailbox/append code leaves me a bit lost, but I've tracked down 
>> the general area of the problem, Ken maybe you can work out what the 
>> right fix is, I don't think it's hard.
>>
>> sieve_fileinto() does the following:
>>
>> 1. cast void * sc -> script_data_t *sd
>> 2. pass "sd->username" as the "authuser" param to deliver_mailbox()
>> 3. deliver_mailbox() calls append_setup() with "authuser" as the "userid" 
>> param
>> 4. append_setup() copys the "userid" parameter into the 
>> "appendstate.userid" struct area
>> 5. during append_commit() "appendstate.userid" is used as the username to 
>> add the seen flag to
>>
>> That all works fine.
>>
>> However, sieve_keep() does the following:
>>
>> 1. cast void *mc -> deliver_data_t *mydata
>> 2. pass "mydata" as the "mydata" param to deliver_local()
>> 3. deliver_local() calls deliver_mailbox(), with "mydata->authuser" as 
>> the "userid" param
>>
>> The value in mydata->authuser is not the username, in fact they're empty:
>>
>> (gdb) p *mydata
>> $29 = { ..., authuser = 0x0, authstate = 0x0}
>>
>> This means the seen flag never gets stored correctly if you're using keep 
>> or the implicit keep, you have to use fileinto.
>>
>> I'm not sure of the best way of fixing this. I can see the obvious 
>> solution (in sieve_keep, cast void * sc -> script_data_t *sd, and copy 
>> authuser and authstate sd to mydata), but that doesn't feel right to me.
>>
>> Ken, do you know what the right solution is here?
>>
>> Rob
>>
>>
>
>
> -- 
> Kenneth Murchison
> Systems Programmer
> Project Cyrus Developer/Maintainer
> Carnegie Mellon University
> 


From lorenzo at sancho.ccd.uniroma2.it  Tue May 27 08:23:23 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo Catucci)
Date: Tue, 27 May 2008 14:23:23 +0200
Subject: new proposed patch for statuscahe off_t size bug
Message-ID: <483BFD3B.2080003@sancho.ccd.uniroma2.it>

This is a follow-up to the patch proposed by John Capo on april, 22.
I'm adding an autoconf check for off_t size, and then use the checked
value to conditionally enable the proposed fix.

Thank you very much, yours

 	lorenzo


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+
-------------- next part --------------
A non-text attachment was scrubbed...
Name: statuscache-off_t.diff
Type: text/x-diff
Size: 2545 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080527/3fe55794/attachment.bin 

From lorenzo at sancho.ccd.uniroma2.it  Tue May 27 08:23:46 2008
From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo Catucci)
Date: Tue, 27 May 2008 14:23:46 +0200
Subject: Small fixes needed for compiling on x86_64 linux
Message-ID: <483BFD52.6020705@sancho.ccd.uniroma2.it>

Please consider updating the config.{guess,sub} files with the enclosed
ones, thereby allowing configure to work on AMD64 and Xeon EM64T linux
sytems.

The enclosed patch does "uncomment" the check for the flags requested to
compile objects to be linked into perl shared-objects; I think that
explicitly whitelisting the check on linux and netbsd platforms shouldn't
do any harm elsewhere...

Thank you very much, yours

 	lorenzo m catucci


+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+
-------------- next part --------------
A non-text attachment was scrubbed...
Name: perl-cccdlflags.diff
Type: text/x-diff
Size: 1200 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080527/bf7a6edb/attachment-0001.bin 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: config.guess
Url: http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080527/bf7a6edb/attachment-0002.ksh 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: config.sub
Url: http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080527/bf7a6edb/attachment-0003.ksh 

From wes at umich.edu  Thu May 29 15:36:55 2008
From: wes at umich.edu (Wesley Craig)
Date: Thu, 29 May 2008 15:36:55 -0400
Subject: ptclient & ldap changes
Message-ID: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>

I have a number of ptclient & ldap bug fixes and improvements to make:

	1) In 2.3.12p2, if ldap_sasl is enabled, user DNs are obtained  
through SASL authN/Z proxying.  This assumes that the LDAP server  
supports authN/Z proxying and that ptclient/ldap has authorization to  
proxy for all users.  I've moved this option under a new  
configuration option, ldap_proxy_authz, since the authZ proxying is  
more or less orthogonal to using SASL for LDAP authN.

	2) Groups have two LDAP configurations, one for populating the  
groups a user belongs to and a second for validating a (new) group  
name.  In 2.3.12p2, those two configurations suffer from non-parallel  
construction.  In particular, ldap_member_method allows both  
"attribute" and "filter", while the ldap_group_* configuration has no  
"_method" configuration, implicitly assuming "filter" instead.  I've  
added a ldap_group_method configuration, with three options,  
"filter", "attribute" and "none".  "none" allows any string that can  
be canonicalized to be used.  "filter" works just like ldap_group_*  
was working -- exactly one DN may be returned.  "attribute" looks for  
at least one DN to be returned.  A correct "attribute" configuration  
searches for the attribute used in ldap_member_attribute.  The  
assumption is that if anyone has the group attribute, it is a valid  
group name.

	3) I changed the default ldap_size_limit to 2.  I also inserted some  
additional checks in the code to specifically look for cases where  
size limit is exceeded.  These may or may not be errors, depending on  
what you're looking for.

	4) I fixed two small bugs in ptloader.c, one where unused memory to  
syslog'd and another where the error message returned from the  
ptloader module isn't null terminated when being passed to auth_pts.c.

Please find the patch attached.  Comments?

:wes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cyrus-imapd-ldap.diff
Type: application/octet-stream
Size: 8406 bytes
Desc: not available
Url : http://lists.andrew.cmu.edu/pipermail/cyrus-devel/attachments/20080529/9b75b88c/attachment.obj 

From igor at ypass.net  Fri May 30 13:50:41 2008
From: igor at ypass.net (Igor Brezac)
Date: Fri, 30 May 2008 13:50:41 -0400
Subject: ptclient & ldap changes
In-Reply-To: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>
References: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>
Message-ID: <48403E71.2030603@ypass.net>

1) I suggest that you keep ldap_sasl for backward configuration 
compatibility.  ldap_sasl  name is used for sasl vs non sasl binds.   
Note that 'ldap_id' needs to have authorization to proxy.

2)  I suppose you can have ldap_group_method: attribute, but this is not 
how ldap data is typically used for groups.  Also, I suggest that 
ldap_group_attribute be used instead of ldap_member_attribute.  As you 
correctly described ldap_group and ldap_member do two different things 
and your implementation would be a bit confusing.   You can possibly 
default ldap_group_attribute to the value of ldap_member_attribute.
I personally do not like ldap_group_method: none, mabe Kan can chime 
in.  This option basically allows for an arbitrary group identifier 
(potentially non existing one) to be assigned to a mailbox.

3) This seems unnecessary, but can you explain a little more?

-Igor

Wesley Craig wrote:
> I have a number of ptclient & ldap bug fixes and improvements to make:
>
>     1) In 2.3.12p2, if ldap_sasl is enabled, user DNs are obtained 
> through SASL authN/Z proxying.  This assumes that the LDAP server 
> supports authN/Z proxying and that ptclient/ldap has authorization to 
> proxy for all users.  I've moved this option under a new configuration 
> option, ldap_proxy_authz, since the authZ proxying is more or less 
> orthogonal to using SASL for LDAP authN.
>
>     2) Groups have two LDAP configurations, one for populating the 
> groups a user belongs to and a second for validating a (new) group 
> name.  In 2.3.12p2, those two configurations suffer from non-parallel 
> construction.  In particular, ldap_member_method allows both 
> "attribute" and "filter", while the ldap_group_* configuration has no 
> "_method" configuration, implicitly assuming "filter" instead.  I've 
> added a ldap_group_method configuration, with three options, "filter", 
> "attribute" and "none".  "none" allows any string that can be 
> canonicalized to be used.  "filter" works just like ldap_group_* was 
> working -- exactly one DN may be returned.  "attribute" looks for at 
> least one DN to be returned.  A correct "attribute" configuration 
> searches for the attribute used in ldap_member_attribute.  The 
> assumption is that if anyone has the group attribute, it is a valid 
> group name.
>
>     3) I changed the default ldap_size_limit to 2.  I also inserted 
> some additional checks in the code to specifically look for cases 
> where size limit is exceeded.  These may or may not be errors, 
> depending on what you're looking for.
>
>     4) I fixed two small bugs in ptloader.c, one where unused memory 
> to syslog'd and another where the error message returned from the 
> ptloader module isn't null terminated when being passed to auth_pts.c.
>
> Please find the patch attached.  Comments?
>
> :wes


From wes at umich.edu  Fri May 30 16:16:54 2008
From: wes at umich.edu (Wesley Craig)
Date: Fri, 30 May 2008 16:16:54 -0400
Subject: ptclient & ldap changes
In-Reply-To: <48403E71.2030603@ypass.net>
References: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>
	<48403E71.2030603@ypass.net>
Message-ID: <4D33877B-28E3-4294-BA99-30E38FE9371F@umich.edu>

On 30 May 2008, at 13:50, Igor Brezac wrote:
> 1) I suggest that you keep ldap_sasl for backward configuration  
> compatibility.  ldap_sasl  name is used for sasl vs non sasl  
> binds.   Note that 'ldap_id' needs to have authorization to proxy.

I left ldap_sasl in place, but removed it's ability to control the  
proxy function.  Instead, it continues to control whether sasl bind  
is used.  Are you suggesting that both ldap_sasl & ldap_proxy_authz  
would be required for proxy to be enabled?  I'd be OK with that,  
presuming that ldap_sasl is indeed required for proxy authz to work.   
The code has a dependency on the proxy authz ldap control, I don't  
recall that it requires sasl.  I suspect not, but I haven't actually  
dug into it.

> 2)  I suppose you can have ldap_group_method: attribute, but this  
> is not how ldap data is typically used for groups.  Also, I suggest  
> that ldap_group_attribute be used instead of  
> ldap_member_attribute.  As you correctly described ldap_group and  
> ldap_member do two different things and your implementation would  
> be a bit confusing.   You can possibly default ldap_group_attribute  
> to the value of ldap_member_attribute.
> I personally do not like ldap_group_method: none, mabe Kan can  
> chime in.  This option basically allows for an arbitrary group  
> identifier (potentially non existing one) to be assigned to a mailbox.

There are two ways to obtain the list of groups a user is a member  
of, corresponding to ldap_member_method "filter" and "attribute".

	filter) A search is executed, typically (uid=%u) in an "ou=groups"  
tree.  The list of DNs found correspond to the groups the user is in.
	attribute) Rather than execute a search, the attributes in the users  
entry are used to form the list of groups the user is in.

The problem is that group name validation doesn't have both models.   
I'm not sure how this discrepancy jibes with your understanding of  
how ldap data is typically used for groups.

As you can see from the patch, I haven't added a  
"ldap_group_attribute" and don't use "ldap_member_attribute".  Both  
ldap_group_method's use ldap_group_filter.  The difference is in how  
the result is treated.

Perhaps you're suggesting that ldap_group_* just be dropped, since  
having two distinct configurations for highly related group  
information is confusing and probably unnecessary.  I'd agree with  
that, but I fixing the current flaw was simpler and more compatible  
than refactoring ptclient/ldap.c entirely.

I don't think the "none" method is any more worrisome than permitting  
someone to assign rights to a group they are not in.

> 3) This seems unnecessary, but can you explain a little more?

It's probably not strictly necessary to change the default size  
limit, but the previous error handling treats "size limit exceeded"  
the same as "not found".  That is a problem, since in some cases the  
"size limit exceeded" is not an error at all.  A very common result  
of the current default & code is that when ldap_member_method is  
"filter", the user gets exactly no groups and no error message  
indicating why.  A more sensible solution would be to make the limit  
10 or 100.  Is there a limit on the number of groups Cyrus allows?   
For ldap_member/group_method of "attribute", a limit of 1 is probably  
smart, since it reduces extraneous traffic.

:wes

> Wesley Craig wrote:
>> I have a number of ptclient & ldap bug fixes and improvements to  
>> make:
>>
>>     1) In 2.3.12p2, if ldap_sasl is enabled, user DNs are obtained  
>> through SASL authN/Z proxying.  This assumes that the LDAP server  
>> supports authN/Z proxying and that ptclient/ldap has authorization  
>> to proxy for all users.  I've moved this option under a new  
>> configuration option, ldap_proxy_authz, since the authZ proxying  
>> is more or less orthogonal to using SASL for LDAP authN.
>>
>>     2) Groups have two LDAP configurations, one for populating the  
>> groups a user belongs to and a second for validating a (new) group  
>> name.  In 2.3.12p2, those two configurations suffer from non- 
>> parallel construction.  In particular, ldap_member_method allows  
>> both "attribute" and "filter", while the ldap_group_*  
>> configuration has no "_method" configuration, implicitly assuming  
>> "filter" instead.  I've added a ldap_group_method configuration,  
>> with three options, "filter", "attribute" and "none".  "none"  
>> allows any string that can be canonicalized to be used.  "filter"  
>> works just like ldap_group_* was working -- exactly one DN may be  
>> returned.  "attribute" looks for at least one DN to be returned.   
>> A correct "attribute" configuration searches for the attribute  
>> used in ldap_member_attribute.  The assumption is that if anyone  
>> has the group attribute, it is a valid group name.
>>
>>     3) I changed the default ldap_size_limit to 2.  I also  
>> inserted some additional checks in the code to specifically look  
>> for cases where size limit is exceeded.  These may or may not be  
>> errors, depending on what you're looking for.
>>
>>     4) I fixed two small bugs in ptloader.c, one where unused  
>> memory to syslog'd and another where the error message returned  
>> from the ptloader module isn't null terminated when being passed  
>> to auth_pts.c.

From igor at ypass.net  Sat May 31 00:06:44 2008
From: igor at ypass.net (Igor Brezac)
Date: Sat, 31 May 2008 00:06:44 -0400
Subject: ptclient & ldap changes
In-Reply-To: <4D33877B-28E3-4294-BA99-30E38FE9371F@umich.edu>
References: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>
	<48403E71.2030603@ypass.net>
	<4D33877B-28E3-4294-BA99-30E38FE9371F@umich.edu>
Message-ID: <4840CED4.70607@ypass.net>

Wesley Craig wrote:
> On 30 May 2008, at 13:50, Igor Brezac wrote:
>> 1) I suggest that you keep ldap_sasl for backward configuration 
>> compatibility.  ldap_sasl  name is used for sasl vs non sasl binds.   
>> Note that 'ldap_id' needs to have authorization to proxy.
>
> I left ldap_sasl in place, but removed it's ability to control the 
> proxy function.  Instead, it continues to control whether sasl bind is 
> used.  Are you suggesting that both ldap_sasl & ldap_proxy_authz would 
> be required for proxy to be enabled?  I'd be OK with that, presuming 
> that ldap_sasl is indeed required for proxy authz to work.  The code 
> has a dependency on the proxy authz ldap control, I don't recall that 
> it requires sasl.  I suspect not, but I haven't actually dug into it.

sasl used to be required for ldap proxy authz, but I do not think this 
is the case any more.  I suggested that both ldap_sasl and 
ldap_proxy_authz do the same thing.
>
>> 2)  I suppose you can have ldap_group_method: attribute, but this is 
>> not how ldap data is typically used for groups.  Also, I suggest that 
>> ldap_group_attribute be used instead of ldap_member_attribute.  As 
>> you correctly described ldap_group and ldap_member do two different 
>> things and your implementation would be a bit confusing.   You can 
>> possibly default ldap_group_attribute to the value of 
>> ldap_member_attribute.
>> I personally do not like ldap_group_method: none, mabe Kan can chime 
>> in.  This option basically allows for an arbitrary group identifier 
>> (potentially non existing one) to be assigned to a mailbox.
>
> There are two ways to obtain the list of groups a user is a member of, 
> corresponding to ldap_member_method "filter" and "attribute".
>
>     filter) A search is executed, typically (uid=%u) in an "ou=groups" 
> tree.  The list of DNs found correspond to the groups the user is in.
>     attribute) Rather than execute a search, the attributes in the 
> users entry are used to form the list of groups the user is in.
>
> The problem is that group name validation doesn't have both models.  

I suppose you could have both methods, but is that really needed?   Each 
group should have one entry in DIT.

> I'm not sure how this discrepancy jibes with your understanding of how 
> ldap data is typically used for groups.
> As you can see from the patch, I haven't added a 
> "ldap_group_attribute" and don't use "ldap_member_attribute".  Both 
> ldap_group_method's use ldap_group_filter.  The difference is in how 
> the result is treated.
> Perhaps you're suggesting that ldap_group_* just be dropped, since 
> having two distinct configurations for highly related group 
> information is confusing and probably unnecessary.  I'd agree with 
> that, but I fixing the current flaw was simpler and more compatible 
> than refactoring ptclient/ldap.c entirely.

It'd be nice if there is a simpler way to configure group (membership 
and validation) in pt/ldap.  :)

>
> I don't think the "none" method is any more worrisome than permitting 
> someone to assign rights to a group they are not in.
>
>> 3) This seems unnecessary, but can you explain a little more?
>
> It's probably not strictly necessary to change the default size limit, 
> but the previous error handling treats "size limit exceeded" the same 
> as "not found".  That is a problem, since in some cases the "size 
> limit exceeded" is not an error at all.  A very common result of the 
> current default & code is that when ldap_member_method is "filter", 
> the user gets exactly no groups and no error message indicating why.  
> A more sensible solution would be to make the limit 10 or 100.  Is 
> there a limit on the number of groups Cyrus allows?  For 
> ldap_member/group_method of "attribute", a limit of 1 is probably 
> smart, since it reduces extraneous traffic.
>

Now looking at the code both ldap_member methods are broken. :( 
- The 'attribute' method stores group DNs in authstate struct which is 
not correct.  Additional search is needed to get group names.
- The 'filter' method, as you pointed out, gets one group at most.  
Changing ldap sitezlimit would fix this issue.

Cyrus does not limit on the number of groups.  I suppose LDAP_NO_LIMIT 
can be used to match other mechanisms, or implement configurable limit 
that can be used in other mechanisms.

It'd be nice if all this complexity can be moved to the ldap 
configuration, similar to the way sasl simplified things for user stuff, 
Dynlist/dyngroups looks interesting...

-Igor

> :wes
>
>> Wesley Craig wrote:
>>> I have a number of ptclient & ldap bug fixes and improvements to make:
>>>
>>>     1) In 2.3.12p2, if ldap_sasl is enabled, user DNs are obtained 
>>> through SASL authN/Z proxying.  This assumes that the LDAP server 
>>> supports authN/Z proxying and that ptclient/ldap has authorization 
>>> to proxy for all users.  I've moved this option under a new 
>>> configuration option, ldap_proxy_authz, since the authZ proxying is 
>>> more or less orthogonal to using SASL for LDAP authN.
>>>
>>>     2) Groups have two LDAP configurations, one for populating the 
>>> groups a user belongs to and a second for validating a (new) group 
>>> name.  In 2.3.12p2, those two configurations suffer from 
>>> non-parallel construction.  In particular, ldap_member_method allows 
>>> both "attribute" and "filter", while the ldap_group_* configuration 
>>> has no "_method" configuration, implicitly assuming "filter" 
>>> instead.  I've added a ldap_group_method configuration, with three 
>>> options, "filter", "attribute" and "none".  "none" allows any string 
>>> that can be canonicalized to be used.  "filter" works just like 
>>> ldap_group_* was working -- exactly one DN may be returned.  
>>> "attribute" looks for at least one DN to be returned.  A correct 
>>> "attribute" configuration searches for the attribute used in 
>>> ldap_member_attribute.  The assumption is that if anyone has the 
>>> group attribute, it is a valid group name.
>>>
>>>     3) I changed the default ldap_size_limit to 2.  I also inserted 
>>> some additional checks in the code to specifically look for cases 
>>> where size limit is exceeded.  These may or may not be errors, 
>>> depending on what you're looking for.
>>>
>>>     4) I fixed two small bugs in ptloader.c, one where unused memory 
>>> to syslog'd and another where the error message returned from the 
>>> ptloader module isn't null terminated when being passed to auth_pts.c.


From wes at umich.edu  Sat May 31 14:43:50 2008
From: wes at umich.edu (Wesley Craig)
Date: Sat, 31 May 2008 14:43:50 -0400
Subject: ptclient & ldap changes
In-Reply-To: <4840CED4.70607@ypass.net>
References: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>
	<48403E71.2030603@ypass.net>
	<4D33877B-28E3-4294-BA99-30E38FE9371F@umich.edu>
	<4840CED4.70607@ypass.net>
Message-ID: <21D95F9A-BAFE-4697-AFDA-0F849934D28C@umich.edu>

On 31 May 2008, at 00:06, Igor Brezac wrote:
> sasl used to be required for ldap proxy authz, but I do not think  
> this is the case any more.  I suggested that both ldap_sasl and  
> ldap_proxy_authz do the same thing.

Perhaps I misunderstand you.  Since SASL authN and proxy authZ are  
more or less completely orthogonal, why would you have them do the  
same thing?  I propose that ldap_sasl control the way bind is done.   
And ldap_proxy_authz is used to control how user DNs are obtained.

> I suppose you could have both methods, but is that really needed?    
> Each group should have one entry in DIT.

If you're using the "attribute" method for group population, the  
group may (should?) have zero entries in the DIT.  That's just  
exactly my point.  The code I've added is a (clever) way to validate  
group names when groups don't have their own DNs -- which is  
frequently the case when membership is recorded in the user's entry.

>> ... fixing the current flaw was simpler and more compatible than  
>> refactoring ptclient/ldap.c entirely.
>
> It'd be nice if there is a simpler way to configure group  
> (membership and validation) in pt/ldap.  :)

True.  Feel free to propose something.  I've fixed the problem I  
have, and improved the code generally.  Further improvements are  
welcome, of course :)

> Now looking at the code both ldap_member methods are broken. :( -  
> The 'attribute' method stores group DNs in authstate struct which  
> is not correct.

You're misunderstanding how ptsmodule_make_authstate_attribute()  
works.  The "attribute" method retrieves the attribute named in  
ldap_member_attribute from the user's DN.  The syntax of that  
attribute is assumed to be in a format suitable for auth_state.  No  
additional search is necessary.

> - The 'filter' method, as you pointed out, gets one group at most.

Again, not exactly.  If you use the default size limit and are in  
more than one group, you get no groups -- size limit is exceeded.   
LDAP_NO_LIMIT might be a useful way to handle this case, but you're  
still left wondering how to handle the case where the server's size  
limit is exceeded.  Do I populate what I got back?  Do I populate  
nothing?

> It'd be nice if all this complexity can be moved to the ldap  
> configuration, similar to the way sasl simplified things for user  
> stuff, Dynlist/dyngroups looks interesting...

sasl simplifies what?  IMHO, using sasl makes almost everything more  
complex.

Cyrus supports two schemes for using groups.  Rather than have more  
or less every configurable LDAP knob visible in imapd.conf, I think  
it would be better if it "just worked".  However, I don't think LDAP  
is very close to that goal as a technology.  A more achievable goal  
for Cyrus's use of LDAP would be posting recipes for the two schemes  
in the wiki.

:wes

From igor at ypass.net  Sat May 31 17:25:13 2008
From: igor at ypass.net (Igor Brezac)
Date: Sat, 31 May 2008 17:25:13 -0400
Subject: ptclient & ldap changes
In-Reply-To: <21D95F9A-BAFE-4697-AFDA-0F849934D28C@umich.edu>
References: <B8A882AE-344D-4D8F-9EBC-67949B3B5E32@umich.edu>
	<48403E71.2030603@ypass.net>
	<4D33877B-28E3-4294-BA99-30E38FE9371F@umich.edu>
	<4840CED4.70607@ypass.net>
	<21D95F9A-BAFE-4697-AFDA-0F849934D28C@umich.edu>
Message-ID: <4841C239.6040807@ypass.net>

Wesley Craig wrote:
> On 31 May 2008, at 00:06, Igor Brezac wrote:
>> sasl used to be required for ldap proxy authz, but I do not think 
>> this is the case any more.  I suggested that both ldap_sasl and 
>> ldap_proxy_authz do the same thing.
>
> Perhaps I misunderstand you.  Since SASL authN and proxy authZ are 
> more or less completely orthogonal, why would you have them do the 
> same thing?  I propose that ldap_sasl control the way bind is done.  
> And ldap_proxy_authz is used to control how user DNs are obtained.
Your patch breaks existing configurations, we usually try to preserve 
configuration compatibility when possible.  Otherwise I am fine with 
your approach.   Maybe automatically set ldap_proxy_authz to true when 
ldap_sasl is turned on and when ldap_proxy_authz is not explicitly 
specified in the config?
>
>> I suppose you could have both methods, but is that really needed?   
>> Each group should have one entry in DIT.
>
> If you're using the "attribute" method for group population, the group 
> may (should?) have zero entries in the DIT.  That's just exactly my 
> point.  The code I've added is a (clever) way to validate group names 
> when groups don't have their own DNs -- which is frequently the case 
> when membership is recorded in the user's entry.

OK.  This makes sense.
>
>>> ... fixing the current flaw was simpler and more compatible than 
>>> refactoring ptclient/ldap.c entirely.
>>
>> It'd be nice if there is a simpler way to configure group (membership 
>> and validation) in pt/ldap.  :)
>
> True.  Feel free to propose something.  I've fixed the problem I have, 
> and improved the code generally.  Further improvements are welcome, of 
> course :)

I was hoping you can make some improvement since I wrote the original 
code.  :)   Unfortunately I have not had time to mess with it in years.  :(

>
>> Now looking at the code both ldap_member methods are broken. :( - The 
>> 'attribute' method stores group DNs in authstate struct which is not 
>> correct.
>
> You're misunderstanding how ptsmodule_make_authstate_attribute() 
> works.  The "attribute" method retrieves the attribute named in 
> ldap_member_attribute from the user's DN.  The syntax of that 
> attribute is assumed to be in a format suitable for auth_state.  No 
> additional search is necessary.

True, although it is broken when values of ldap_member_attribute are 
represented with DN.  You are probably right, not worth messing with.

>
>> - The 'filter' method, as you pointed out, gets one group at most.
>
> Again, not exactly.  If you use the default size limit and are in more 
> than one group, you get no groups -- size limit is exceeded.

If you are a member of one group, it works.  We are saying the same 
thing.  Bottom line and I think you'll agree, the code needs improvement.

> LDAP_NO_LIMIT might be a useful way to handle this case, but you're 
> still left wondering how to handle the case where the server's size 
> limit is exceeded.  Do I populate what I got back?  Do I populate 
> nothing?
>

I suppose LDAP_NO_LIMIT is lesser of two evils.  It seems impractical 
for a user to be member of several hundred groups...

>> It'd be nice if all this complexity can be moved to the ldap 
>> configuration, similar to the way sasl simplified things for user 
>> stuff, Dynlist/dyngroups looks interesting...
>
> sasl simplifies what?  IMHO, using sasl makes almost everything more 
> complex.

I respectfully disagree, maybe it is just me.  :)  I suppose for 
openldap admins it may be more complex. 
In terms of cyrus imap admin, all they should need to know is ldap 
username/pass.  They should not need to have any knowledge of DIT, 
filters, bind dn, etc...

>
> Cyrus supports two schemes for using groups.  Rather than have more or 
> less every configurable LDAP knob visible in imapd.conf, I think it 
> would be better if it "just worked".  However, I don't think LDAP is 
> very close to that goal as a technology.  A more achievable goal for 
> Cyrus's use of LDAP would be posting recipes for the two schemes in 
> the wiki.
>

Agreed, but I do not think it is far off.

-Igor