From developer@opengroupware.org Fri Jul 20 21:05:48 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Fri, 20 Jul 2007 16:05:48 -0400 Subject: [OGo-Developer] big resource hog in SoObject Message-ID: <03177f7fda4b3c9540628f393c480447@mozzarella> Hi Helge and all, In SOGo, most data objects implement a "lookupName:inContext:acquire:" method. This method first calls its super equivalent in the case a web method with the required name would exist. The problem is that the super method also invokes "toOneRelationShipKeys", to make sure the name is available. It's ok for small directories but not for LDAP directories with 1000s of entries. Because each time a contact card is requested, all entries are scanned again, while it should be the directory controller object to check whether the name is available or not... So my request would be to remove that part between line 138 and 144 of SoObject.m. Would that be reasonable? Wolfgang From developer@opengroupware.org Sat Jul 21 11:04:49 2007 From: developer@opengroupware.org (Helge Hess) Date: Sat, 21 Jul 2007 12:04:49 +0200 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: <03177f7fda4b3c9540628f393c480447@mozzarella> References: <03177f7fda4b3c9540628f393c480447@mozzarella> Message-ID: <0285FF8D-F8E4-4FE8-93B0-86C9F408D5AA@opengroupware.org> On 20.07.2007, at 22:05, Wolfgang Sourdeau wrote: > In SOGo, most data objects implement a > "lookupName:inContext:acquire:" method. This method first calls its > super equivalent in the case a web method with the required name > would exist. > The problem is that the super method also invokes > "toOneRelationShipKeys", to make sure the name is available. > It's ok for small directories but not for LDAP directories with > 1000s of entries. Because each time a contact card is requested, > all entries are scanned again, while it should be the directory > controller object to check whether the name is available or not... > > So my request would be to remove that part between line 138 and 144 > of SoObject.m. > > Would that be reasonable? I have no strong opinion on that, but the impression that the current behaviour is quite a good (convenient) generic default behaviour because it allows you to drop in EO objects w/o additional work. The toOneRelationshipKeys thing is for EOModel/CoreData applications, and you rarely have 10+ toOne relationships. Besides that I also have the impression that toOneRelationshipKeys should never ever return 1000+ objects?! That is, the default implementation of that method is obviously a no go for your LDAP controller? I suppose your controller might want to expose the children as a single toManyRelationshipKey. Or probably not at all. I don't know why SOGo would ever call toOneRelationshipKeys on an object (outside -lookupName). WebDAV has its own EOFetchSpecification based methods which can be overridden for performance (see ZideStore) and the web interface can always properly qualify searches in the proper action. Summary: I would be interested on ZNeKs opinion on that. We can drop it, but I *think* that your issue is at a different place (the way your object maps to KVC). Thanks, Helge -- Helge Hess http://www.helgehess.eu/ From developer@opengroupware.org Mon Jul 23 10:26:40 2007 From: developer@opengroupware.org (=?ISO-8859-1?Q?Marcus_M=FCller?=) Date: Mon, 23 Jul 2007 11:26:40 +0200 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: <0285FF8D-F8E4-4FE8-93B0-86C9F408D5AA@opengroupware.org> References: <03177f7fda4b3c9540628f393c480447@mozzarella> <0285FF8D-F8E4-4FE8-93B0-86C9F408D5AA@opengroupware.org> Message-ID: --Apple-Mail-1-717169931 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Hi all, On 21.07.2007, at 12:04, Helge Hess wrote: >> The problem is that the super method also invokes >> "toOneRelationShipKeys", to make sure the name is available. >> > Summary: I would be interested on ZNeKs opinion on that. We can > drop it, but I *think* that your issue is at a different place (the > way your object maps to KVC). I had a similar issue once and I wasn't really convinced that the default behaviour is appropriate in all situations. We might want to introduce a method which triggers the behaviour, so it can be altered in special cases like the one Wolfgang describes? Cheers, Marcus -- Marcus Mueller . . . crack-admin/coder ;-) Mulle kybernetiK . http://www.mulle-kybernetik.com Current projects: http://www.mulle-kybernetik.com/znek/ --Apple-Mail-1-717169931 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1 Hi all,

On 21.07.2007, at 12:04, Helge Hess = wrote:

The problem = is that the super method also invokes "toOneRelationShipKeys", to make = sure the name is available.
Summary: I would be = interested on ZNeKs opinion on that. We can drop it, but I *think* that = your issue is at a different place (the way your object maps to = KVC).

I had a similar issue once and I = wasn't really convinced that the default behaviour is appropriate in all = situations. We might want to introduce a method which triggers the = behaviour, so it can be altered in special cases like the one Wolfgang = describes?

=

Cheers,


=A0=A0Marcus


--=A0

Marcus Mueller=A0=A0.=A0=A0.=A0=A0.=A0=A0crack-admin/coder = ;-)

Mulle = kybernetiK=A0=A0.=A0=A0http://www.mulle-kybernetik.com

Current = projects:=A0http://www.mulle-kybernetik= .com/znek/


 =

= --Apple-Mail-1-717169931-- From developer@opengroupware.org Mon Jul 23 11:02:28 2007 From: developer@opengroupware.org (Helge Hess) Date: Mon, 23 Jul 2007 12:02:28 +0200 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: References: <03177f7fda4b3c9540628f393c480447@mozzarella> <0285FF8D-F8E4-4FE8-93B0-86C9F408D5AA@opengroupware.org> Message-ID: On 23.07.2007, at 11:26, Marcus M=FCller wrote: > We might want to introduce a method which triggers the behaviour, =20 > so it can be altered in special cases like the one Wolfgang describes? Not yet another method please :-) Its just whether we want to change =20 the default behaviour or not. An implementor can easily change the behaviour by overriding -=20 lookupName:inContext: (eg in SOGoObject) in any case. Greets, Helge --=20 Helge Hess http://www.helgehess.eu/ From developer@opengroupware.org Mon Jul 23 12:07:27 2007 From: developer@opengroupware.org (Adam Tauno Williams) Date: Mon, 23 Jul 2007 07:07:27 -0400 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: <03177f7fda4b3c9540628f393c480447@mozzarella> References: <03177f7fda4b3c9540628f393c480447@mozzarella> Message-ID: <1185188847.4162.7.camel@aleph.whitemice.org> > In SOGo, most data objects implement a "lookupName:inContext:acquire:" > method. This method first calls its super equivalent in the case a web > method with the required name would exist. > The problem is that the super method also invokes > "toOneRelationShipKeys", to make sure the name is available. > It's ok for small directories but not for LDAP directories with 1000s > of entries. Because each time a contact card is requested, all entries > are scanned again, while it should be the directory controller object > to check whether the name is available or not... This is a side note; but if an LDAP query is returning 1000s of entries you have another problem as well, your DSA is not configured with sane limits. A limit of 500 objects should be more than adequate for ANY application. If you focus is scalability (as in SOGo) this is a serious problem. As an LDAP administrator I'd be pretty unhappy [understatement] with any application that pulled thousands of objects at a time. Also no application should every produce as generic a filter as to match thousands (or even hundreds) of objects as such a result set it almost never actually valid/useful. In OpenLDAP you can configure limits like - limits anonymous size.soft=512 size.hard=1024 size.unchecked=32767 time.soft=10 time.hard=60 limits users size.soft=2048 size.hard=2096 size.unchecked=32767 time.soft=15 time.hard=60 - these are documented in man slapd.conf If an application really does need huge results sets [indicating, IMHO, that it is fundamentally broken] then it should use paging controls to deal with DSA limits. I'd recommend, at least, that your default filter be one that matches no objects; that way you don't pull anything unless the application takes the time to set a real filter. > So my request would be to remove that part between line 138 and 144 of > SoObject.m. > Would that be reasonable? From developer@opengroupware.org Mon Jul 23 12:54:34 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Mon, 23 Jul 2007 07:54:34 -0400 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: Message-ID: On 2007-07-23 06:02:28 -0400 Helge Hess w= rote: > On 23.07.2007, at 11:26, Marcus M=FCller wrote: >> We might want to introduce a method which triggers the behaviour, so= it=20 >> can be altered in special cases like the one Wolfgang describes? >=20 > Not yet another method please :-) Its just whether we want to change = the=20 > default behaviour or not. > An implementor can easily change the behaviour by overriding -=20 > lookupName:inContext: (eg in SOGoObject) in any case. Yes but if you override it, you lose the features that it has. Such as w= eb method resolution... In my opinion, I am not sure the check it does is appropriate at that le= vel. I consider that it's my object that should do the appropriate check= s... W. From developer@opengroupware.org Mon Jul 23 13:05:51 2007 From: developer@opengroupware.org (Helge Hess) Date: Mon, 23 Jul 2007 14:05:51 +0200 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: References: Message-ID: On 23.07.2007, at 13:54, Wolfgang Sourdeau wrote: > Yes but if you override it, you lose the features that it has. Such > as web method resolution... Web methods are resolved by the SoClass? Just call if ((o = [[self soClass] lookupName...]) != nil) return [o bindToObject:self inContext:_ctx]; Thats *really* easy. In fact its _very_ common that one wants to change the lookup method to speed up the lookup process. Eg in ZideStore object ids are always numeric and method names are never, allowing you to avoid either lookup, the same goes for the SOGo mailer. But again: I think the real flaw is that toOneRelationshipKeys is used incorrectly in the SOGo LDAP controller. It should simply not return all IDs of the LDAP folder. Adam's post is quite valid in that context too. Thanks, Helge -- Helge Hess http://www.helgehess.eu/ From developer@opengroupware.org Mon Jul 23 13:35:34 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Mon, 23 Jul 2007 08:35:34 -0400 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: Message-ID: <997fa7993c32bbaa2537271495e6ed22@Maestro.local> On 2007-07-23 08:05:51 -0400 Helge Hess wrote: > On 23.07.2007, at 13:54, Wolfgang Sourdeau wrote: >> Yes but if you override it, you lose the features that it has. Such >> as web >> method resolution... > > Web methods are resolved by the SoClass? Just call > > if ((o = [[self soClass] lookupName...]) != nil) > return [o bindToObject:self inContext:_ctx]; > > Thats *really* easy. > In fact its _very_ common that one wants to change the lookup method > to > speed up the lookup process. Eg in ZideStore object ids are always > numeric > and method names are never, allowing you to avoid either lookup, the > same > goes for the SOGo mailer. Ok. Then I could place this in SOGoObject. But it's a little pornish don't you think? > But again: I think the real flaw is that toOneRelationshipKeys is > used > incorrectly in the SOGo LDAP controller. It should simply not return > all IDs > of the LDAP folder. Adam's post is quite valid in that context too. For LDAP, that is another issue which I partly agree with... If the user expects to list all the entries of an LDAP directory, then it should happen... But for regular use, I do plan to limit the search results. And if you "connect" to an ldap addressbook from the web interface, nothing will show by default (same behaviour as Thunderbird). You *have* to do a search to get something. Wolfgang From developer@opengroupware.org Mon Jul 23 13:47:29 2007 From: developer@opengroupware.org (Helge Hess) Date: Mon, 23 Jul 2007 14:47:29 +0200 Subject: [OGo-Developer] big resource hog in SoObject In-Reply-To: <997fa7993c32bbaa2537271495e6ed22@Maestro.local> References: <997fa7993c32bbaa2537271495e6ed22@Maestro.local> Message-ID: <07E22B3D-872D-4549-B7E1-E8B3EFF70DB8@opengroupware.org> On 23.07.2007, at 14:35, Wolfgang Sourdeau wrote: > Ok. Then I could place this in SOGoObject. But it's a little > pornish don't you think? I don't know what you mean by 'pornish'. SoObjects are controller objects and -lookupName: is the primary method to resolve URLs. Now how child URLs are resolved obviously depends on the controller, which is why its highly application specific and the method almost always needs to be overridden. >> But again: I think the real flaw is that toOneRelationshipKeys is >> used incorrectly in the SOGo LDAP controller. It should simply >> not return all IDs of the LDAP folder. Adam's post is quite valid >> in that context too. > For LDAP, that is another issue which I partly agree with... If the > user expects to list all the entries of an LDAP directory, then it > should happen... Yes, but toOneRelationshipKeys is the wrong API for retrieving all items. This should be done by a proper EODataSource/ EOFetchSpecification based method. Which you need anyways because you would usually want to qualified searches ... > But for regular use, I do plan to limit the search results. And if > you "connect" to an ldap addressbook from the web interface, > nothing will show by default (same behaviour as Thunderbird). You > *have* to do a search to get something. Personally I think that it is more user friendly to run a limited query (say first ten records sorted by lastname) even if the user didn't type anything. This is inexpensive and the user can see that the store contains items. Further I would always limit searches to something like 50 items per default. If the query then hits the limit, I would show that to the user and then give him the option to double/triple/ignore the limit. Usually he will just restrict the qualifier further. This way you avoid bothering the LDAP server unnecessarily, and the user will have results much faster. Thanks, Helge -- Helge Hess http://www.helgehess.eu/ From developer@opengroupware.org Fri Jul 27 16:37:07 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Fri, 27 Jul 2007 11:37:07 -0400 Subject: [OGo-Developer] SOPE and multipart/form-data Message-ID: Hi, I have a form with multipart/form-data set as encoding type. This form contains dynamically generated file input fields. However I see no method in SOPE to manage the file's filename nor their mime-types, which forces me to reparse the HTTP request in a way that I consider not really clean. Is there an alternative to that? Wolfgang From developer@opengroupware.org Fri Jul 27 16:43:40 2007 From: developer@opengroupware.org (Helge Hess) Date: Fri, 27 Jul 2007 17:43:40 +0200 Subject: [OGo-Developer] SOPE and multipart/form-data In-Reply-To: References: Message-ID: <2F6FC493-5797-48A3-9C04-64F2B3F9C4B5@opengroupware.org> On 27.07.2007, at 17:37, Wolfgang Sourdeau wrote: > I have a form with multipart/form-data set as encoding type. This > form contains dynamically generated file input fields. However I > see no method in SOPE to manage the file's filename nor their mime- > types, which forces me to reparse the HTTP request in a way that I > consider not really clean. ---snip--- @interface WOFileUpload : WOInput ... WOAssociation *filePath; // dispostion 'filename' ... ---snap--- Helge -- Helge Hess http://www.helgehess.eu/ From developer@opengroupware.org Fri Jul 27 18:27:01 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Fri, 27 Jul 2007 13:27:01 -0400 Subject: [OGo-Developer] SOPE and multipart/form-data In-Reply-To: <2F6FC493-5797-48A3-9C04-64F2B3F9C4B5@opengroupware.org> Message-ID: > ---snip--- > @interface WOFileUpload : WOInput > ... > WOAssociation *filePath; // dispostion 'filename' > ... > ---snap--- > > Helge > When writing "generated dynamically", I was meaning that is was generated in javascript, not by SOPE. W. From developer@opengroupware.org Fri Jul 27 20:01:49 2007 From: developer@opengroupware.org (Helge Hess) Date: Fri, 27 Jul 2007 21:01:49 +0200 Subject: [OGo-Developer] SOPE and multipart/form-data In-Reply-To: References: Message-ID: <86E5D46A-D63F-4FAD-A390-1A07CA1131BD@opengroupware.org> On 27.07.2007, at 19:27, Wolfgang Sourdeau wrote: >> ---snip--- >> @interface WOFileUpload : WOInput >> ... >> WOAssociation *filePath; // dispostion 'filename' >> ... >> ---snap--- >> Helge > When writing "generated dynamically", I was meaning that is was > generated in javascript, not by SOPE. You could still use WOFileUpload to process requests, but anyways. If you really want to handle uploads manually, you just need to do it like WOFileUpload does? It basically accesses the underlying NGHttpRequest, WebObjects' WORequest has no API for retrieving multipart information. Greets, Helge -- Helge Hess http://www.helgehess.eu/ From developer@opengroupware.org Sat Jul 28 22:43:53 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Sat, 28 Jul 2007 17:43:53 -0400 Subject: [OGo-Developer] SOPE and multipart/form-data In-Reply-To: <86E5D46A-D63F-4FAD-A390-1A07CA1131BD@opengroupware.org> Message-ID: > You could still use WOFileUpload to process requests, but anyways. If > you > really want to handle uploads manually, you just need to do it like > WOFileUpload does? It basically accesses the underlying > NGHttpRequest, > WebObjects' WORequest has no API for retrieving multipart > information. I don't know how I could intercept WOFileUpload instances from inputs generated from javascript/DOM? Regading the NGHttpRequest way, that's the way I am doing it, but I was hoping to use something higher level... Wolfgang From developer@opengroupware.org Sat Jul 28 22:47:30 2007 From: developer@opengroupware.org (Wolfgang Sourdeau) Date: Sat, 28 Jul 2007 17:47:30 -0400 Subject: [OGo-Developer] NGMime and encoding Message-ID: Hi, I am intercepting a bunch of text fields from a multipart/form-data encoded form. However it seems Firefox doesn't produce mime-type neither provides encoding information for this. This makes NGMime return a default of "text/plain; charset=us-ascii", while the data really is UTF-8 (which can be deduced from the encoding used to produce html content). Since NGMime is also used to decode mail parts, I am not sure how I can patch NGMime to accept an encoding set externally when the data source is an HTTP stream... Wolfgang