http://git.samba.org/?p=vl/samba.git/.git;a=shortlog;h=idmap

Hi!

$SUBJECT is an effort to simplify and streamline the idmap
implementation in winbind.

Main changes:

The cache directly uses gencache, so you can watch and
delete cached mappings with "net cache".

"idmap domains" is gone, the parent winbind figures out the
list of trusted domains itself. If it becomes aware of a new
trusted domain, it looks in smb.conf whether a special
"idmap config" style configuration is around. If so, then it
tells the winbind idmap child about that fact in the child
request.domain_name field. The range parameter in idmap
config settings is mandatory, the parent winbind uses it to
direct the unix id to sid mappings correctly.

The default domain is set via "idmap backend", "idmap config
foo : default = yes" is gone. It is highly recommended if
not required that this backend is writable for not specially
configured unknown trusted domains.

"idmap alloc backend" defaults to "idmap backend". "idmap
alloc backend" is only required to set explicitly if you
have a special unix id source, id mappings will always be
written to the "idmap backend". "idmap alloc config : range"
is gone, as this must match the "idmap [ug]id" valid for
"idmap backend" anyway, we now only use that.


From my point of view this is upwards compatible with
pre-3.0.25 configs, post-3.0.25 configs will have to set
their default domain via "idmap backend".

Volker

PGP SIGNATURE
Version: GnuPG v1.4.5 (GNU/Linux)


ZTAFscIsa+4BGVub8EtJ9n4=
=wyTa
PGP SIGNATURE

Thu, Jul 17, 2008 at 09:04:33AM -0400, simo wrote:
Thu, 2008-07-17 at 14:56 +0200, Volker Lendecke wrote:
Thu, Jul 17, 2008 at 08:52:20AM -0400, simo wrote:
Please don't check this in until I have a chance to take a look.
I am leaving today for vacation so that may take a while.

When will you be back?

August

Don't think we can wait this long for "European" length
vacations :-). You're a New Yorker now, things move faster.

I'm going to review this and if it's good, let's get it
in. Sorry Simo but we can't wait that long for code
reviews.

Jeremy.

Thu, 2008-07-17 at 12:38 +0000, Volker.Lendeckea wrote:
Hi!

$SUBJECT is an effort to simplify and streamline the idmap
implementation in winbind.

Main changes:

The cache directly uses gencache, so you can watch and
delete cached mappings with "net cache".

"idmap domains" is gone, the parent winbind figures out the
list of trusted domains itself. If it becomes aware of a new
trusted domain, it looks in smb.conf whether a special
"idmap config" style configuration is around. If so, then it
tells the winbind idmap child about that fact in the child
request.domain_name field. The range parameter in idmap
config settings is mandatory, the parent winbind uses it to
direct the unix id to sid mappings correctly.

The default domain is set via "idmap backend", "idmap config
foo : default = yes" is gone. It is highly recommended if
not required that this backend is writable for not specially
configured unknown trusted domains.

"idmap alloc backend" defaults to "idmap backend". "idmap
alloc backend" is only required to set explicitly if you
have a special unix id source, id mappings will always be
written to the "idmap backend". "idmap alloc config : range"
is gone, as this must match the "idmap [ug]id" valid for
"idmap backend" anyway, we now only use that.

I have been taking a brief look at the patches, there a few things I
like very much.

Things I like:
- use of gencache, so that we have one less set of cache functions
- check of the cache in the main winbind daemon which removes one step
when we have a cache hit (this was long due and planned, thanks for
doing this)

Things I don't like or find controversial:
- I'd like an explanation of the reasons this patch cut back on some
options and functionality and not just the description of a new behavior
(so that people can understand why and not just what).

- the interface to query multiple ids at the same time (exp. multiple
SIDs to uid/gid) going away (I really do not like this one).
This interface was built as an optimization to observed performance
issues at login time. In some complex domain setups the PAC can contain
an enormous amount of SIDs (more than 2000 for a single user), when
trying to resolve these SIDs the 1 by 1 check become a real bottleneck
which can delay a single authentication by many seconds (and keeps
winbindd busy a lot too). The reason is that with cold caches we may get
as much as 2000 separate SID lookups against the domain controller (slow
LDAP/RPC calls) and if idmap_ad or idmap_ldap are used another 2000 sid
to uid/gid calls. This can take many seconds, and if multiple users are
logging in at the same time the situation becomes really bad.
The solution was implemented in 2 parts: 1) the idmap cache (with
negative caching to handle cases where idmap_ad (or other readonly
backend is used) and there is no mapping available), 2) allow winbind to
query multiple IDs at the same time so that a single LDAP call could be
made to solve all idmap mappings instead of 2000 separate calls.
The interface was later disabled by Jerry for concerns I cannot recall
at this moment, but I think it should be re-enabled and not definitely
eliminated.

- the patch is incompatible with 3.0.25 and 3.2.0 configurations not
only at the smb.conf level

- the patch allows only the default domain to be written to, as the
allocator backend is not allowed a different range from the default
backend, and uid2sid checks are bound to the id range anyway.
The original designed allowed multiple backends to be written to, so
that a single server could write to any backend (given appropriate
credentials).
The initial idea was to allow each backend to have it's own allocator so
that both the domain SID and the uid/gid range could be used to
determine the domain to query.
After Jerry's request a decision was made to 'simplify' the allocation
procedure and have only one single allocator.
Still I didn't want to loose the ability to allocate IDs both in the
local domain and in any trusted domain that allowed our server to add id
mappings, so domains were identified only by name and SID and uid ranges
were only optional and used as filters (mainly for read-only domains).
A scenario where this would be useful is a server joined to an AD domain
using tdb (or a local ldap) for local mappings and with one of the
trusted domains being a Samba domain that shared mappings via LDAP. In
this scenario (with a single allocator) you want to allocate mappings on
the ldap server so that ids are unique on all servers and yet save
mappings both in the local tdb (for local mappings) and on the LDAP
server when the mapping is about that trusted domain.
This was always a thing I didn't like much but it seemed to simplify
things.
I'd agree on a patch that removes the single allocator and instead
allows any backend that is not read only to have it's own allocator
functions. The only problem I see is that existing configuration would
break if a range is made mandatory and range filtering enforced when
UIDs are spread among multiple writable backends. But the simplification
would probably be worth the minor number of migrations and would also
speed up some lookups when many trusted domain are present.
Multiple allocation backends present a problem though, you cannot have a
simple "gimme a new id function", you now will need to always specify
for what domain you want it (think about passdb and group mapping for
things like BUILTIN\Administrators on a member server), so that will
require some thinking and some special casing I guess, but it is
entirely doable.

- the 'default' parameter going away if my own domain is not what I want
to be the default. If the samba server using idmap_rid is joined to a
'minor' domain in a forest or more complex setup and you want to keep
one LDAP server with company-wide mappings (where most other trusted
domains are stored), then you may want to have mappings, by default, to
go on the LDAP server that is referenced by trusted domains
configurations.
I guess here too we can settle on loosing this functionality and forcing
people to migrate their server configurations and idmapping storage
(except that sometimes this is not something admins have really the
power to do) and tell them they have to put everything on one LDAP
server, although I don't like forcing these choices on admins, therefore
I'd prefer to salvage these options if at all possible.

There are other things that have been changed where I really do not have
an opinion in favor or against they are just different because the code
have different premises and goals.

Simo.

Hi!

Please find a bugfix in $SUBJECT.

Mon, Jul 21, 2008 at 03:45:13PM +0000, Simo wrote:
Things I don't like or find controversial:
- I'd like an explanation of the reasons this patch cut back on some
options and functionality and not just the description of a new behavior
(so that people can understand why and not just what).

- the interface to query multiple ids at the same time (exp. multiple
SIDs to uid/gid) going away (I really do not like this one).

That code was not used so far. I understand it is
potentially very useful, but with the idmap cache in
gencache we can grant r/o-access to it from smbd directly.
So assuming a long positive idmap cache time (i.e. a week),
it should be much less of a problem.

And, we can for sure re-add the code again.

- the patch is incompatible with 3.0.25 and 3.2.0 configurations not
only at the smb.conf level

The incompatible change I see is the params argument to the
module init call and the cleanup of the idmap_domain
structure. I do think the idmap_domain structure was really
overloaded, the modules should not have to know if they are
default, readonly or initialized. If they have to know this,
they can put their stuff into the private_data pointer.

- the patch allows only the default domain to be written to, as the
allocator backend is not allowed a different range from the default
backend, and uid2sid checks are bound to the id range anyway.

Yes, and this is a deliberate choice. Nobody I asked, and I
did ask quite a few people (except you obviously ;-))
understood the interrelationship between the different
backend types, ranges, and so on. So I think a severe
simplification is really necessary, potentially sacrificing
some flexibility.

Still I didn't want to loose the ability to allocate IDs both in the
local domain and in any trusted domain that allowed our server to add id
mappings, so domains were identified only by name and SID and uid ranges
were only optional and used as filters (mainly for read-only domains).
A scenario where this would be useful is a server joined to an AD domain
using tdb (or a local ldap) for local mappings and with one of the
trusted domains being a Samba domain that shared mappings via LDAP. In
this scenario (with a single allocator) you want to allocate mappings on
the ldap server so that ids are unique on all servers and yet save
mappings both in the local tdb (for local mappings) and on the LDAP
server when the mapping is about that trusted domain.

From my point of view this is a bit too much flexibility.
Forcing users to assign the same gid for BUILTIN\Users on
all hosts participating in the LDAP based idmap is something
I could find arguments for, given that the configuration
becomes a lot simpler and more predictable this way.

- the 'default' parameter going away if my own domain is not what I want
to be the default. If the samba server using idmap_rid is joined to a
'minor' domain in a forest or more complex setup and you want to keep
one LDAP server with company-wide mappings (where most other trusted
domains are stored), then you may want to have mappings, by default, to
go on the LDAP server that is referenced by trusted domains
configurations.

With the code I propose the backend configured with "idmap
backend" must be writable. You can easily then configure
"idmap config <mydomain: backend = rid" where <mydomain>
is the domain you are joined to. Tested that.

Volker

PGP SIGNATURE
Version: GnuPG v1.4.5 (GNU/Linux)


LL+d6MEtGTdUu2wan38cluA=
=iEkq
PGP SIGNATURE