/interpreter/tags/at2-build060407/TODO.txt

  1=== TODO ===
  2
  3------------------
  4Conceptual issues:
  5------------------
  6
  7Meta-circular AmbientTalk:
  8==========================
  9
 10Goal: write as much as possible AT/2 code in AT/2 itself. A lot of NAT* classes, especially AG* classes
 11can be written in AT/2 itself. Most NAT* classes are plain AT/2 objects with a custom mirror that overrides
 12only a few operations.
 13
 14The only problem is that this makes the implementation much slower as a lot of NAT classes will then become
 15NATMirages. We should make our implementation of NATMirage as efficient as possible. We need some kind of
 16'partial behavioural reflection' in order to pay only for the reflection that we need.
 17This means that as much meta_* methods should remain hard-wired even for certain mirages, because they are not overridden
 18by the mirror.
 19
 20For those NAT classes that need access to native Java objects, we might be able to use the Symbiosis.
 21As such we may be able to confine the AT/2 kernel to only a handful of classes (NATNil, NATObject, JavaObject)
 22
 23Prototypes:
 24===========
 25
 26We need to augment the standard AT/2 library with a namespace full of prototypes.
 27E.g. especially exception prototypes such that native exceptions can be raised from within the language.
 28at.exceptions.interpreter
 29
 30The Standard Library:
 31=====================
 32
 33Written entirely in AmbientTalk itself. Accessible via 'lobby.at'
 34We need:
 35
 36- at.init: an initialization file with useful default semantics for most things currently in the OBJLexicalRoot?
 37Should also contain isNumber(x), isTable(x) , ... natives.
 38- at.collections: a small collections library (binary tree, hashtable, vector, list, queue, set) implementing
 39a uniform interface (iteration, collection, filtering, insertion, deletion, lookup)
 40- at.unit: a unit testing framework along the lines of JUnit, the unit testing framework of Ruby etc:
 41make a sub-object of the TestCase object and implement some methods whose name starts with test*. Running the program should run the tests.
 42- at.mirrors: prototypical mirrors on all kinds of language values?
 43- at.exceptions: prototype exception objects for native exception types that can be newed and raised.
 44
 45Identity and equality:
 46======================
 47
 48Next to the == operator, which defines equality through identity, we should introduce
 49an operator = which defines structural equality (cfr. eq? versus equal? in Scheme).
 50e.g. [2,3] is not == to another [2,3] table, but [2,3] = [2,3]
 51
 52== is already implemented and falls back on Java's equals method, which in turn falls back on Java's == operator.
 53Structural equality implies deep equality.
 54For 'leaf' objects like symbols, numbers, etc., deep equality is identity.
 55For 'compound' objects like tables, all elements of the tables must be deep equal.
 56
 57The same goes for objects. Objects can be considered as maps from symbols to values, so
 58two objects are equal if they map the same symbols to equal values. The problem here
 59is that performing deep equality like this is both
 60A) very costly
 61B) dangerous because it is a graph-traversal
 62(have to keep track of already visited objects? even for tables, consider: t[1] := t)
 63
 64Another kind of comparison between objects that could be useful is 'substitutability'
 65For example, obj1 '<:' obj2 if they have the same field names and method selectors
 66(i.e. the 'range' of their conceptual 'map' is similar) such that obj2 can safely
 67'replace' obj1.
 68
 69One can then define:
 70boolean NATObject.base__opltx__opcol_(ATObject other) {
 71  foreach field | method selector of this {
 72    if (other.meta_respondsTo(selector) {
 73       continue;
 74    } else {
 75       return false;
 76    }
 77  }
 78  return dynamicParent_.base__opeql_(other.meta_getDynamicParent());
 79}
 80
 81Note that structural equivalence of objects is defined in terms of their public fields/methods
 82+ those of their dynamic parents. Lexical parents are not taken into consideration, as they are
 83considered 'private' to the object.
 84
 85Symbiosis:
 86==========
 87
 88- First-class constructors: jClass.new or jObject.new should wrap a JavaConstructor class,
 89such that casting works for constructors as well.
 90=> Implementation can be sped up a bit using caching. Currently, only methods are cached, could also
 91cache Constructors if we introduce a JavaConstructor class? This could also be used to solve problem of
 92casting for constructors. E.g. if aClass.new returns a JavaConstructor which is a special kind of JavaClosure,
 93then we can do aClass.new.cast(Type1, Type2)(a, b)
 94
 95- Solve overloading by being able to annotate a method invocation => future work?
 96
 97- methods whose selector contains '_' in Java: we should allow _ in AT identifiers.
 98Only disadvantage is that Java identifiers with '_' will be displayed using ':' in AT.
 99However, an AT selector will not undergo such conversions when remaining within AT.
100Can be made even cleaner if the _ -> : conversion is only performed for downSelector,
101and for symbiosis introduce another method named 'javaToATSelector'
102
103Keywords and Optional/Varargs:
104==============================
105
106Remember that we wanted some kind of keyworded message sends that
107were matched against a kind of 'regular expression' for stuff like
108try-catch and when-catch, such that e.g. the following invocations
109all trigger the same language construct:
110when: f becomes: { |v| ... }
111when: f becomes: { |v| ... } due: 1000
112when: f becomes: { |v| ... } catch: { #Exc1 |e| ... } catch: { #Exc2 |e| ... }
113
114This could be done using optional and vararg parameters with keywords:
115def when: future becomes: block due: time := default catch: @blocks {
116  ...
117}
118This method actually has a selector when:becomes:[due:](catch:)*
119and regexp matching could be used to match invocation with definition.
120
121We really need this kind of semantics if we want to make optional
122arguments and varargs work for keywords.
123
124Meta-send (send-to-mirror):
125===========================
126
127Consider introducing another message passing operator (.?) which sends a message
128to an object's mirror rather than to the object itself.
129
130obj.?msg(args) == (reflect: obj).msg(args).base
131
132The advantage is that the meta-variant does not require the creation of temporary mirrors.
133
134The reflect: variant is much more expensive; it requires creating a mirror on obj
135and a mirror on the result, which are usually immediately discarded.
136
137Next to efficiency considerations, the meta-send is also a more concise way
138of sending messages to mirrors.
139
140Update: now that we have message annotations, we could use them to express this
141operation as obj.msg(args)@Meta
142
143The only problem is that, if such messages are sent *directly* to the mirror,
144this would circumvent the mirror factory (e.g. the use of reflect: to obtain a mirror
145on an object).
146
147Annotations:
148============
149
150Annotations or stripes or types are necessary for multiple reasons:
151- For object classification (e.g. allow objects to categorize themselves
152into domain-specific categories, or tell to the interpreter that they represent
153a table, a function, a number, etc. -> for better coercion)
154- For symbiosis with Java (e.g. objects should tell AmbientTalk which Java
155interfaces they implement)
156- For exception handling (catch only exception objects of a certain kind)
157- For service discovery: need a topic hierarchy to allow interpreter
158to rapidly filter/route discovery events)
159
160All of these require 'subtyping' semantics
161(i.e. an annotation/stripe/type can have 0 or more supertypes and a subtype is substitutable for a supertype)
162
163Apart from the above requirements to tag *objects*, sometimes it would be
164interesting to also tag *fields*, *methods*, *message sends* or even *closures*
165Here, these annotations represent declarative directives to the meta-level, which
166e.g. alter field access, method invocation, message dispatch
167- fields: proxy objects that implement get and set
168- methods: proxy objects that implement apply and applyInScope
169- message: proxy implementing sendTo
170- closure: ?
171Especially interesting for introducing proxies (cfr. the 'function transformations' from dPico)
172(custom fields, method wrappers)
173A requirement for these annotations is that they should carry arguments, like:
174obj.msg(a,b,c)#annotation(arg)
175e.g. obj<-msg(a,b,c)#delay(1000)
176
177Such annotations could e.g. be used to solve overloading problems with the Java symbiosis:
178javaobject.method(a#type1, b#type2, c#type3)
179or one could pass an object tagged with a correct type
180
181It would be nice to integrate annotations consistently into the language and be able to
182use them both as directives to the meta-level *and* as types for objects.
183
184Transform annotations into message sends?
185def #foo x := 5 => def x := foo().decorateField(`x, 5) // of decorateField(<field object>)
186=> value? should be a custom field object!
187def #bar(1) meth(args) { ... } => def meth := bar(1).decorateMethod(<method object>) => method
188obj.msg(args)#foo => obj <+ foo().decorateMessage(<msg object>) => message
189{ #foo |args| body } => foo().decorateClosure({ |args| body }) => closure
190
191object: #foo { ... } => decorateParameter
192object: { #foo ... } => decorateClosure
193object: { ... }#foo => decorateParameter
194
195Asynchronous function invocation:
196=================================
197
198Introduce the shorthand syntax fun<-(a,b,c) to denote fun<-apply([a,b,c])
199such that remote references to closures are more easily invokable.
200
201Isolates and Import:
202====================
203
204Import internally makes use of ClosureMethod objects to implement the delegate methods
205that are imported into the host. These objects are currently not isolates, so when an isolate
206uses import, it will probably lose its delegate methods when being passed by copy.
207
208Perhaps the methods should be actually copied when being imported into an isolate?
209The problem with this is that this loses lexical scoping rules:
210a copied variable ref 'x' has a different binding!
211
212Mirrors:
213========
214
215Intercessive mirrors are always tied to a particular 'base' object.
216The default intercessive mirror is named 'mirrorroot' and is an object
217that understands all meta_* operations, implementing them using default semantics.
218
219It can be thought of as being defined as follows:
220
221def mirrorroot := object: {
222  def base := object: { nil } mirroredBy: self // base of the mirror root is an empty mirage
223  def init(b) {
224    base := b
225  }
226  def invoke(@args) { <default native invocation behaviour> }
227  def select(@args) { <default native selection behaviour> }
228  ...
229} stripedWith: [ Mirror ]
230
231This object can then simply be extended / composed by other objects to deviate from the default semantics.
232Hence, 'mirrors' are simply objects with the same interface as this mirrorroot object: they should be
233able to respond to all meta_* messages and have a 'base' field.
234
235When a mirror is cloned, its base field is *shared* with the clone. However, the base object will still be
236tied to the original mirror. Therefore, the default clone: operator is implemented as follows:
237
238def clone: obj {
239  if: (is: obj stripedWith: Mirror) then: {
240    reflect: (clone: obj.base)
241  } else: {
242    (reflect: obj).clone()
243  }
244}
245
246In other words: when a mirror is cloned, the mirage is cloned instead (recursively) and a mirror on the new mirage is returned.
247Cloning mirages also clones the corresponding mirror - and re-initializes the cloned mirror's mirage to the cloned mirage, so there
248the bidirectional link between mirror and mirage is not broken.
249
250There is another way to assign mirrors to mirages, by means of the object:mirroredBy: primitive:
251
252def mirage := object: {
253  ...
254} mirroredBy: aMirror
255
256Internally, this operation is translated into
257aMirror.new(<uninitialized mirage>)
258which is:
259aMirror.meta_clone().invoke("init", [<uninitialized mirage>])
260
261For regular mirrors, which are IS-A children of the mirrorroot, this leads to the invocation of 'init' on aMirror,
262and eventually also to the invocation of 'init' on OBJMirrorRoot. This init implementation sets the base field to point
263to the new mirage, and initializes the 'mirror' field of the mirage to the dynamic receiver of 'init'.
264From that point on, the mirage is considered initialized and can be used.
265An 'uninitialized' mirage is one whose mirror is assigned to 'nil'.
266When explicitly making a new instance of a mirror, OBJMirrorRoot checks whether the object
267is receives in its init method is indeed a NATMirage, and whether its mirror is != nil.
268
269The only problem currently with object:mirroredBy: is that the mirror is *implicitly* instantiated,
270which also means that the arguments to new are always [base]. This is annoying if the mirror should be
271parameterized with other properties as well. Perhaps introduce
272object: { ... } mirroredBy: aMirror initialize: [ extra args ]
273
274Lexical Scope vs External Definitions:
275======================================
276
277When performing e.g. def o.x := 5
278and 'x' was already lexically visible in the scope of 'o', should this
279definition become visible, or should all references to 'x' in the scope of o
280remain bound to the lexically visible x?
281
282Currently, the external definition will 'shadow' the lexically visible 'x'.
283Using lexical addressing might solve such issues.
284
285----------------------
286Implementation issues:
287----------------------
288
289Meta-circular AmbientTalk:
290==========================
291
292Related to a proper meta-circular implementation of AmbientTalk is to allow as much custom AT objects in
293place of implementation-level objects. That is, we should remove as much .asNative...() casts from the code
294as possible. Each such cast precludes use of custom AT objects. Sometimes, the casting is inevitable (e.g. 1+2),
295sometimes, it is not. E.g.: rather than casting an ATObject to a NATTable and looping over its elements_, one could write:
296if (obj instanceof NATTable) {
297  // perform immediate native behaviour
298  // loop over ((NATTable) obj).elements_ with for loop
299} else {
300  obj.base_asTable().base_each(new NativeClosure(this) {
301    // perform native behaviour inside of a native closure's apply method
302  });
303}
304
305Native method sanity check:
306===========================
307- zorgen dat alle AT interfaces alleen nog base_ / meta_ methoden hebben die InterpreterException throwen
308en die alleen AT args of convertible args returnen / nemen
309violators:
310NATText meta_print() throws InterpreterException;
311alle isXXX() methods => returnen boolean
312ATObject base_object_mirroredBy_(ATClosure code, NATIntercessiveMirror mirror) throws InterpreterException;
313en varianten: nemen expliciet een NATIntercessiveMirror ipv een ATMirror
314
315Static Variables:
316=================
317
318We must ensure the integrity of all static variables in the AT/2 implementation with respect
319to race conditions! Static variables are shared by multiple actors. This means that they
320should definitely be final + that the datastructure they point at is immutable. If not,
321e.g. the string pool for symbols, access should be synchronized!
322
323
324Native Method Refactoring:
325==========================
326
327Goal is to get rid of most methods from the JavaInterfaceAdaptor and Reflection classes,
328and to make native AmbientTalk methods accessible without having to perform a reflective
329Java method invocation.
330
331Primary goal: to remove all reflective invocations of base_ methods from the implementation.
332Instead, we want to represent native methods more closely as AmbientTalk methods.
333Like this:
334- we don't need to convert AT selectors into Java identifiers,
335- we can reuse AT's parameter passing mechanism for native methods (e.g. varargs)
336- we don't have to perform a reflective Java method invocation.
337
338To change:
339- every NAT class should have a static final HashTable that represents
340the method dictionary of all objects of that NAT type.
341
342- at startup time, this HashTable should be filled with the native
343methods of that NAT type. Native methods are represented as anonymous
344classes that implement a kind of native_apply method, which is parameterized
345by:
346- the Java 'this' (because native methods are static and shared by all
347instances, they are selfless, which should be of the proper NAT type
348[- the AmbientTalk 'self' (necessary if a native method needs late binding))]
349- a call frame that binds formals to actuals
350
351- meta_invoke on native objects (e.g. NATNil) should query the class-level
352hashtable for the appropriate native method corresponding to the selector,
353and apply the native method. Alternatively, meta_invoke could become a large
354switch statement, but this is harder to maintain + would lead to code duplication
355in between meta_invoke, meta_delegate, meta_select.
356
357Consider for example the method '+' in NatNumber:
358class NATNumber extends NATNumeric {
359  private final int javaValue_;
360  private static final HashTable _NATNumber_METHODS_ = new HashTable();
361  static {
362    _NATNumber_METHODS_.put(AGSymbol.alloc("+"), new NativeMethod("+",new ATObject[]{AGSymbol("other")}) {
363      public ATObject native_apply(NATNumber ths, ATObject slf, NATCallframe args) {
364        ATObject other = args.meta_select(AGSymbol.alloc("other")); // alternatively, do args.at(0)
365        return ths.base_plus(other.meta_asNumber());
366      }
367    }
368  }
369  public ATNumber base_plus(ATNumber other) { ... }
370}
371
372Alternatively, to speed up lookup of native methods,
373each native method's selector could be assigned a number, and
374lookup can occur in O(1) by storing the methods in an array where
375the indices correspond to the selector's number.
376
377
378Introspective mirrors need to be adapted similarly:
379They should implement native methods with selectors "invoke", "select", etc.
380e.g.
381new NativeMethod("invoke",new ATObject[] { AGSymbol("selector"), AGSymbol("args") }) {
382  public ATObject native_apply(NATMirror m, ATObject self, NATCallframe args) {
383     return up(m.base.meta_invoke(args.at("selector").meta_asSelector(), args.at("args").meta_asTable()));
384  }
385}