/doc/development-notes.txt

  1== Form handling / Validation ==
  2
  3Add validations and form handling.
  4
  5	event({submit, ...}, Context).
  6	
  7Result is javascript, might be redirect javascript.
  8
  9Validation:
 10	- attach validations to data of the to be posted element
 11	- sign with nonce and md5
 12	- name="bla" add to the post:  "z_v=bla:email:nonce:aa1b8db21157fae46b72e8731f4023da"
 13
 14  wf:wire(continueButton, nameTextBox, #validate { validators=[
 15    #is_required { text="Required." },
 16    #custom { text="Must start with 'Rusty'.", tag=some_tag, function=fun custom_validator/2 }
 17  ]}),
 18
 19becomes:
 20
 21	<input type="text" name="email" id="bla" value="" />
 22	{% validate id="bla"
 23		type={presence}
 24		type={email}
 25		type={not_in_use delegate="xxx"}
 26	%}
 27
 28When you don't supply the delegate then either:
 29- there is a validator module with the name validator_xxxx (where xxxx = validation type)
 30- there is a validator(some_extra_validate, ..) in the resource
 31
 32You can access:
 33
 34	z_context:q()
 35	z_context:q_validated()
 36	
 37Javascript needed:
 38
 39	add-some-validation(#bla),
 40	$(#bla).data("z_validation", "pickled-postback-data");
 41
 42TODO:
 43	- add live validation refs in javascript  - done.
 44	- add onsubmit handler - done.
 45	- add onsubmit handler after possible form insertion - done.
 46	- add pickled info per field in the data - done.
 47	- collect all pickled data on submit - done.
 48	- collect form fields on submit - done.
 49	- send ajax with form to postback (either generic "form" or postback wired to form) - done.
 50	- in postback handler: 
 51		- catch form submit - done.
 52		- check all pickled validation handlers -done.
 53			- postback the invalid fields + error messages - done.
 54		- call event(submit, ...) of resource (or delegate - as wired to the form) - done.
 55		- add javascript z_validation_error(Id, Error) - done.
 56	- add more erlang validations - done.
 57	- add redirect action - done.
 58
 59
 60== Redo the context vars - should not be in context to prevent duplicating on message passing ==
 61
 62Possible solutions:
 63
 64	- Let the template server return the template and evaluate the template in the request thread (do not pass context around) - done.
 65
 66Will need some context vars for lookup of @include variables (which is ok).
 67
 68== Inline Templates ==
 69
 70Make it possible to translate inline templates, i.e. not from a file.
 71
 72== Depcache ==
 73
 74Make a depcache server, now the scomp server uses its own simple cache.
 75It is better to make one with dependencies, so that the scomps can give MaxAge _and_ dependencies
 76
 77TODO:
 78	- check if is better to store big data in dict (avoid copying)
 79
 80== User support ==
 81
 82Need support for users.  
 83
 84One user can have:
 85	- multiple sessions, with each multiple pages
 86	- leaves an unique id on a user agent (different per ua - store in database)
 87	- has unprotected info (nickname, shopping cart)
 88	- has private info (real name, address, e-mail)
 89	- is_anonymous flag
 90
 91When an anonymous user logs on then the data of the anonymous user gets merged into the data of the user.
 92For this we need to have a merge strategy per variable - maybe put it in the varname?
 93Strategies per user variable:
 94	- merge_bag
 95	- merge_set
 96	- replace_userdata
 97	- keep_userdata (default)
 98	- transient (will not be stored in db)
 99 
100User:define_var(Name, Attrs)
101
102Let modules 'hook' into the user merging / startup routines and solve the variables problem in that way?
103
104User has three property tables (which are reflected in the user process state):
105	1. Public (just user cookie is enough)
106	2. Protected (must have an autologon, or authenticated)
107	3. Private (must have recently authenticated < 30 minutes)
108
109Timestamps on user/person:
110	1. Created
111	2. Last Visit
112	3. Last Authenticated (only in user process)
113
114States of user/person:
115	1. Anonymous (expiry date - when do we delete this person)
116	2. User (which does not assume verified - just that this is a known user)
117
118Cookie on user-agent:
119	1. zpuid - per user agent different
120	2. in database coupled
121		- coupled with person info
122		- last visit timestamp
123		- expiry timestamp
124		- autologon check, as a expiry timestamp
125
126Flow:
127
128	- On first visit:
129		1. Create new user, flag as anonymous user
130		2. Add cookie to user, set cookie "zpuid" - valid for 10 years or so - done.
131		3. Name of user is empty, no details known except for last visit
132		4. Set autologon of cookie to false - an anonymous user can't logon
133	- On next visit: - done.
134		1. Grab user from db, using zpuid cookie
135		2. If no such user -> handle as first visit - done.
136		3. Set 'last visit' of user to now()
137		4. If autologon status set, mark user in session as logged on (protected stuff is visible)
138 
139	- On user creation:
140		1. Create new user
141		2. Send user an e-mail with account details
142		3. Log on as the new user (see below)
143	- On user logon:
144		1. Find user record with username/password (or openid)
145		2. Set autologon status of zpuid cookie to checkbox value
146		3. If current user is anonymous -> Copy/merge public information over to new user
147		4. Change zpsid and zpuid (safety measure)
148	- On user logoff:
149		1. Set user process state to 'public' (locking protected and private properties)
150
151
152== Scomps - code change ==
153
154Catch code changes for the scomps so that they can be re-initialised.
155	- Ask question about this on erlang-questions group
156
157== Templates - production switch ==
158
159Options to disable the modification checks during production.
160
161== Templates - custom_tags ==
162
163Test & check include paths for the custom_tags
164
165== Templates - dependencies + change 'depend' to 'vary' ==
166
167Add:
168
169	{% vary variable %}
170
171This adds a 
172
173	-varies(['variable']).
174
175property to the translated template. This will be used by the include scomp.
176
177Change depend parameter of scomps to 'vary' - done.
178
179== Google chart ==
180
181Check code, adapt calling interface to something that will work with our template system - done.
182
183== LOG ==
184
185Add rotating logger to the webmachine logger
186
187== Webmachine / MochiWeb ==
188
189Integrate webmachine and mochiweb in our source tree, check makefiles.
190
191== Translations and filter applications ==
192
193A translation is of the form:
194
195 {_ english text _}
196
197or
198
199  {% _ "english" nl="nederlands" fr="fran??ais" %}
200
201and is translated by the grammar as:
202
203 {trans, [{en,"english text"},{nl,"nederlands"},...]}
204
205which is translated as:
206
207 % ...
208 Language = erlydtl_runtime:fetch_value(language, Variables),
209 % ...
210 z_trans:trans({trans, [{en, "english text"}]}, Language)
211 % ...
212
213We always need to apply an extra runtime filter on a value to make sure that the value is not a {trans} from a database field.
214
215Also add the prefix '_' operator to strings:
216
217	_"hello world"|escape
218	
219or:
220
221	{{ title|default:_"untitled" }}
222
223
224TODO:
225	- Add the z_i18n directory for i18n modules - done
226	- Add the {{ _ en="english text" }} construct - done
227	- Delay z_trans:trans application for parameters of filters - nice to have - not done yet
228	- Modify erlydtl filters so that optional {trans} parameter (and argument) are translated - done
229	- Move language to #context{} - done.
230	- Add all gettext functionality (get .po files, use them, refetch them)
231	- Add scanning of different strings in the templates and erlang code (to .pot file)
232
233
234== Add Context as parameter to the template routines ==
235
236Pass the context to the different lookup routines, separate the context from the variables.
237Especially the erlydtl_runtime:value_lookup() routine must have the context as an extra parameter.
238
239TODO:
240	- modify erlydtl compile - done.
241	- modify erlydtl runtime lookups - done.
242	- modify z_scomp to correctly handle contexts - done.
243	- modify scomp behaviour + implementations - done.
244	- modify calls to z_template, add vars parameter - done.
245	
246
247== Handling of resources in Erlang code and templates ==
248
249(All below is done, some small variations in the real implementation vs. the notes below.)
250
251We want to have something like the anyMeta $thing in our templates and Erlang code.
252Possible actions on this 'thing' (resource or m_rsc)
253
254  - Get property
255  - Get list of media attached
256  - Get list of all objects (optionally filtered by predicate)
257  - Get list of all subjects (optionally filtered by predicate)
258
259Updates of properties / object / subject are done with separate function calls to m_rsc.
260
261		-module(m_rsc).
262		-export([
263			rsc/0,
264			exists/1, 
265			is_readable/2, is_writeable/2, is_owner/2, is_ingroup/2, is_me/2
266			p/3, 
267			op/2, o/2, o/3. o/4,
268			sp/2, s/2, s/3. s/4,
269			media/2, media/3
270		]).
271		
272		rsc() -> fun(Id, _Context) -> #rsc{id=Id} end.
273
274		exists(Id) -> true | false.
275		is_readable(Id, Context) -> true | false.
276		is_writeable(Id, Context) -> true | false.
277		is_owner(Id, Context) -> true | false.
278		is_ingroup(Id, Context) -> true | false.
279		is_me(Id, Context) -> true | false.
280
281		%% Perform access control checks, return 'undefined' on an error
282		%% Unknown properties will be checked against the predicates, returns o(Predicate).
283		p(Id, Predicate, Context) -> 
284			undefined | Value
285
286		%% Return a list of all edge predicates of this resource
287		op(Id, Context) ->
288			[].
289
290		%% Used for dereferencing object edges inside template expressions
291		o(Id, Context) ->
292			fun(P) -> o(Id, P, Context) end.
293
294		o(Id, Predicate, Context) ->
295			{rsc_list, []}.
296			
297		o(Id, Predicate, Index, Context) ->
298			#rsc{id=SomeId}.
299			
300		%% Return a list of all edge predicates to this resource
301		sp(Id, Context) ->
302			[].
303
304		%% Used for dereferencing subject edges inside template expressions
305		s(Id, Context) ->
306			fun(S) -> s(Id, P, Context) end.
307
308		s(Id, Predicate, Context) ->
309			{rsc_list, []}.
310			
311		s(Id, Predicate, Index, Context) ->
312			#rsc{id=SomeId}.
313
314		media(Id, Context) ->
315			[].
316
317		media(Id, Index, Context) ->
318			undefined | MediaPropList.
319
320During template evaluation we can cache rsc proplists in the process dictionary.
321It is also possible to pre-assign resources as a batch in the controller.
322
323In template:
324
325	{{ rsc[Id].title }}  (can return a trans #trans record {trans, [{en,"english text"},{nl,"..."}]})
326
327or just:
328
329	{{ rsc[Id] }}
330
331which will give #rsc{id=Id}, which will be evaluated to its id by the renderer.
332
333The following expressions are the same:
334
335	{{ rsc[Id].author.name }}
336	{{ rsc[Id].o.author.name }}
337	{{ rsc[Id].o.author[1].name }}
338
339	rsc => fun(Id) -> #rsc{id=Id} end.
340	Id  => #rsc{}
341	o   => fun (P) -> {rsc_list, []} end.
342	author => {rsc_list, [#rsc{}|..]} 
343	1 => #rsc{}
344	name => Value 
345	
346In erlang this will be:
347
348	m_rsc:p(m_rsc:p(Id, author, Context), name, Context);
349
350The expression
351
352	{{ rsc[Id].author[1] }}
353
354gives as result:
355
356	#rsc{id=AuthorId}
357
358Where the expression:
359
360	{{ rsc[Id].author }}
361
362gives as result:
363
364	[ #rsc{id=AuthorId1}, #rsc{id=AuthorId2}, ... ]
365
366The m_rsc:p() function returns the predicate of the first #rsc{} in a #rsc list.
367
368	p(#rsc{id=Id}, Predicate) -> 
369		do_something;
370	p(Id, Predicate) when is_integer(Id) ->
371		p(#rsc{id=Id}, Predicate);
372	p([R|_], Predicate) ->
373		p(R, Predicate);
374	p([], _Predicate) -> undefined;
375	p(undefined, _Predicate) -> undefined;
376
377
378== (Product) Properties ==
379
380triplet:
381
382	(property-id, optional-property-value-id, optional-term)
383
384property ->
385
386	(group-id, property-text, [(property-value-id, text)])
387
388group ->
389
390	(text)
391
392
393For template ->
394
395	[
396		{
397			group-text,
398			[
399				{property-text, property-value-text},
400				...
401			]
402		},
403		...
404	]
405	
406== Module/ how they extend the system ==
407
408Modules:
409
410	module_indexer:
411	- listen to {scomp/ template/ action/ validator/ model} notifications, maps them to the ones
412	  found in the module folders - done.
413	- listen to {module_activate/ module_deactivate} notifications, rescan the list of scomps (etc)
414	  after receiving, also instructs the dispatcher to reload the dispatch rules - done.
415	- listen to {module_rescan} message, as send from the dev module to force a rescan when in
416	  development mode - done.
417
418	module_admin
419	- for overview of modules, activate/deactivate modules in the admin - done.
420
421	admin
422	- basic admin controllers - done.
423	
424	shop
425	- the webshop site - done.
426
427	google_chart
428	- The google chart scomp, and also chart_pie and chart_pie3d - now still in mod_base
429
430	base
431	- all normal buttons, drag/drop, validators etc. - done.
432
433	dropbox
434	- handle new images in the upload folder, move & publish them - still a gen_server in src/support
435
436	pivot
437	- handles extraction of texts for indexing rsc entries - still a gen_server in src/support
438
439	
440When looking up an item:
441
442	Send a notification {item-kind, item-name, Args, Context}.
443
444		{scomp, button, [{text,"Click Me"}], Context}
445	
446	The module manager will receive this notification and check against the known list of items.
447	Other modules can also hook into the notification manager to intercept the message and return
448	an alternative.  This can be on the basis of the arguments or the supplied context.
449	- done.
450	
451Get a list of all admin menu items:
452
453	{% for title, url in m.notify.admin_menu %}
454
455	This sends a notification {admin_menu} to all modules.
456	The resulting list is filtered from empty results and returned.
457	
458	- not done yet - will be implemented when needed.
459
460Call a scomp in a module:
461
462	{% scompname arg=value %}
463	
464	The module manager scans the scomp directory of every module, so it can do a mapping 
465	from the name of scomps found there to scomps requested.
466	Scomp names should be:  scomp_<modulename>_<scompname>.erl
467
468	It uses the -module_priority() attribute of the module or the scomp to sort the scomps, enabling
469	a scomp to be overruled by another module.
470
471	- above is done.
472
473	LATER:
474	
475	A fallback is to send a notification {scomp, scompname, [Arg]}, the first answer that
476	is non-undefined will be returned.  
477
478	The answer can be:
479	- an iolist() for direct result
480	- an atom, comprising the name of the module that should handle the scomp
481	- a context, which should be updated with the result of the scomp rendering
482	- a tuple {iolist(), Context}, comprising the rendered result of the scomp
483
484Call all scomps by all modules:
485
486	{% all scompname arg=value %}
487	- done.
488
489Include a template from a module:
490
491 	{% include "templatename.tpl" %}
492
493	The module manager scans the templates directory in all modules. It can map a requested name to
494	a found template.  <modulename>_<templatename>.tpl
495
496	It uses the -module_priority() attribute to sort the templates, enabling a template to be overruled
497	by another module.
498	- done.
499	
500Include all templates by all modules:
501
502	{% all include "templatename.tpl" %}
503	- done.
504	
505Refer to an action in a module:
506
507	The module manager scans the actions directory of the every module, so it can do a mapping
508	from the name of the actions.  action_<modulename>_<actionname>.erl
509
510	It uses the -module_priority() attribute of the module or the action to sort the actions, enabling
511	an action to be overruled by another module.
512	- done.
513	
514Refer to a validator in a module:
515
516	The module manager scans the validators directory of the every module, so it can do a mapping
517	from the name of the validators.   validator_<modulename>_<validator>.erl
518	- done.
519	
520Refer to a model in a module:
521
522	The module manager scans the model directory of every module. It maps the names found to the
523	names requested.  m_<modulename>_<modelname>.erl
524	
525	- Will not do this for now, you should specify unique names for all models and address them in that way.
526	Reason: not needed at this moment and extra messaging overhead to find the correct model.
527
528