APAM dependency management
The traditional resource management strategy is to first gather all the resources needed by an application before starting it. Unfortunately, in our context, between time t0 at which a service s is started and time t1 at which it needs a service provider P, many things may occur:
- P may be non-existing at t0, but created before t1;
- P may be unavailable or used at t0 but released before t1;
- a provider of P (say p1) may be available at t0 but at t1 it is another provider (say p2) that is available.
We call resolution the process by which a client finds the service provider (an instance) it requires.
In APAM, a dependency is defined towards a component (specification, implementation or instance) or a resource (an interface or a message) defined by their name, constraints and preferences.
If the dependency is defined toward a component, the resolution consists first in finding that component and then to select one of its members satisfying the constraints and preferences, and recursively until to find the instance(s). If the dependency is defined toward a resource, the resolution consists in finding a component providing that resource and satisfying the constraints and preferences, and recursively until to find the instance(s). If no instance satisfies the constraint but an implementation is available, an instance is created; otherwise the resolution fails.
The components are found either in the platform (the currently running services), or in a repository, local or distant (OBR, Maven, ...). Since the component description is the same in all repositories, including the platform, the same constraints and preferences apply indifferently in all repositories. The available repositories are per composite. If found in a repository, the selected component is transparently deployed and instantiated; therefore, for the client developer, it makes no difference if the component is found in the machine or in any repository. Conceptually, all the components are in the machine (like between the virtual memories and the physical memory).
Nevertheless it is always possible for a resolution to fail i.e. no convenient implementation or instance can be found, in that case, by default, null is returned to the client i.e. the client code must check its variable before any use, which is relevant only if the dependency is optional. On all the other cases, the client would like to assume that its variable is always conveniently initialized. The strategy in this case is controlled by the "fail" property associated with dependencies. For example:
<dependency specification="S2" field="s2" id="fastS2"
fail= "wait"|"exception"|"optional" exception="fr.imag...failedException" />fail="wait" means that if the resolution fails, the client current thread is halted. When a convenient provider appears, the client thread is resumed with its dependency resolved against that provider. Therefore, the client code can always rely on a satisfactory resolution, but may have to wait.
fail="exception" means that, if the resolution fails, the "ResolutionException" is thrown. The source code is supposed to catch that exception.
fail="optional" means that the dependency is considered as optional, and a null value is returned in the variable if the dependency cannot be resolved. The programmer is supposed to test the variable value if there is any risk that the resolution may fail. By default, fail="optional".
exception="Exception class" means that, if the dependency fails, the associated exception is thrown. At compile time, the Exception class must be present in one of the bundle used by Maven during the compilation of the component. A execution time, the class must be loaded.
If, for any reason (failure, disconnection, ...) the instance used by a dependency disappears, APAM simply removes the wire, and a new resolution of that dependency will be intended at the next use of the associated variable. It means that dynamic substitution is the default behavior.
A "simple" dependency is associated with a simple variable in the Java code. At any point in time, the variable points to zero (null) or one provider. A multiple dependency is associated with a variable that is a collection i.e. an "array", a "Set", a "Vector" or a "List". Such a dependency therefore leads to a set of service providers. When the dependency is resolved for the first time, the dependency is associated with all the instances implementing the required resources, available at the time of resolution. If none are available, one is instantiated if possible, the resolution fails otherwise. The collection is replaced by another one each time a new provider "appears" or "disappears"; therefore the variable always contains the actual set of available providers. Warning: the variable may change while the Java program is using the previous value.
<dependency specification="S3Compile" id="S3Id" multiple="true">
<interface field="fieldS3" multiple="true"/> <!— multiple is useless -->The multiple attribute is very useful only for specification dependencies, since there is no other way, at that level, to know. For implementations, the field type (Collection or not) indicates if the dependency is multiple or not. If the field is a collection, the attribute multiple can be missing, it is assumed to be true, it can be set to true, but it cannot be false.
Once the dependency resolved, any new instance (of the right type) appearing in the system is automatically added to the set initially computed; similarly, each time an instance disappears, it is removed from the set of instances. This even can be captured in the program, if callbacks are indicated:
<dependency field="fieldT" added="newT" removed="removedT" />In this example, if fieldT is a set of type T, the Java program must contain a method newT and removeT (names are fully arbitrary):
Set<T> fieldT;
public void newT (T t) {}
// or
public void newT (Instance inst) {}
public void removedT () {}
// or
public void removedT (Instance inst) {}The method newT must have as parameter either an object of type T, or an object of type Instance (fr.imag.apam.Instance). This method is called each time an object (of type T) is added in the set of references, this object is the parameter. Similarly, the method removeT is called each time an object is removed from the set; it may have the APAM instance object as parameter (warning: it is an isolated object without a real instance inst.getServiceObject()==null)
About messages, the newM1 method is called each time a new provider is added in the set of the M1 message providers, and removedM1 is called when an M1 provider is removed.
A complex dependency is such that different fields and messages are associated with the same provider instance. The provider must implement a specification, and the different fields must reference the different resources defined by that specification.
<dependency specification="S3Compile" id="S3Id">
<interface field="fieldS3" />
<message method="mes1" />
<interface field="field2S3" />
</dependency>In the example, the dependency S3Id is a dependency toward one instance of the specification S3Compile. That instance is the target of fields fieldS3 and field2S3, and the provider of message mes1. For dependencies with cardinality multiple, all variables are bound to the same set of service providers (internally, it is the same array of providers). It means that that dependency is resolved once (when the first field is accessed), and if it changes, it changes simultaneously for all fields.
Following our metamodel, a component provides resources (interfaces or messages) and a dependency can be defined against interfaces or messages. Therefore a component can be a message provider, or a message requester.
A message provider must indicate in its declaration header, as for interfaces, the type of the provided messages, and for implementations, the associated fields (see example above).
<specification name="S2" interfaces="apam.test.S2"
messages="apam.test.M1, apam.Test.M2">
...
<implementation name="S2Impl" specification="S2"
push="producerM1, producerM2" interfaces="apam.test.AC">The S2Impl implementation should contain the methods producerM1 and producerM2:
public M1 producerM1 (...) { return m1; }Each time the producer calls the producerM1 method, APAM considers that a new M1 message is produced. There is no constraint on the method producerM1 parameters, but it must return an M1 object. A dependency can be defined against messages in a similar way as interfaces, but methods instead must be indicated in the case of push interactions or a java.util.Queue field in the case of pull interactions, as in the following examples:
<dependency pull="queueM1" />
<dependency field="fieldS2" />
<dependency specification="S2Compile" >
<interface field="anotherS2" />
<message push="getAlsoM1" />
<message pull="anotherQueueM1" />
</dependency>
<dependency push="gotM2" />
<dependency pull="queueM2" />The first line is a simple declaration of a message dependency; analyzing the source code it is found that queue M1 is a field of the type java.util.Queue that has a message of type M1 as a paramterType and therefore is associated with the message M1 dependency. The associated Java program should contain:
Set<S2> fieldS2 ;
S2 anotherS2 ;
Queue<M1> queueM1;
Queue<M1> anotherQueueM1;
public void getAlsoM1 (M1 m1) { ... }
Queue<M2> queueM2;
public void gotM2 (M2 m2) { ... }Queue fields are special: Queue are instantiated by APAM at the first time call, then APAM place all then new messages inside them. If there is no new M1 value available the queue is empty, and if there is no producer the queue is null (see resolution policy).
At the first call to these queues, the corresponding M1producers are resolved and connected to the queue. If the dependency is multiple, all the valid M1 producers will be associated to the queue, otherwise a single producer is connected. In this case, as for usual dependencies, it is the client that has the initiative to get a new value. We call it the pull mode.
A producer my can also declare methods that return a set of messages:
public Set<M1> producerM1 (...) { ... }When these methods are called, APAM will consider that all the returned objects are provided messages.
For consumers, the declared method is void (push interactions), with a message type as parameter (M2 here), this method will be called by APAM each time a message of type M2 is available. In this case it is the message provider that has the initiative to call its client(s). The connection between client and provider is established at the first call by the provider to its produceM2 method. In the example, the method gotM2 will be called each time an M2 message is produced by one of the valid M2 producers.
In the previous examples, the raw data of type M1 and M2 is received by the clients. If more context is required, the injected methods or Queue can declare Message<M1> instead of M1; Message being a generic type defined in APAM that contains a M1 value and information about the message: producer id, time-stamp, and so on.
For multiple message dependencies, as for interfaces, it is possible to be aware of the "arrival" and "departure" of a message provider:
<dependency push="getM1" added="newM1Producer" removed="removedM1Producer " />With the associated methods, as shown above for interfaces.
<dependency specification="S3Compile" id="S3Id">
<interface field="fieldS3" />
<constraints>
<implementation filter="(apam-composite=true)" />
<instance filter="(&(testEnum*>v1,v2,v3)(x=6))" />
<instance filter="(&(A2=8)(MyBool=false))" />
</constraints>
<preferences>
<implementation filter="(x=10)" />
<instance filter="(MyBool=false)" />
</preferences>
</dependency>
<definition name="testEnum" type="v1, v2, v3, v4, v5" value="v3" />In the general case, many provider implementations and even more provider instances can be the target of a dependency; however it is likely that not all these providers fit the client requirements. Therefore, clients can set filters expressing their requirements on the dependency target to select. Two classes of filters are defined: constraints and preferences. Filters can be defined on implementations or instances. Constraints on implementation are a set of LDAP expression that the selected implementations MUST ALL satisfy. An arbitrary number of implementation constraints can be defined; they are ANDed. Similarly, constraints on instance are a set of LDAP expression that the selected instances MUST ALL satisfy. An arbitrary number of instance constraints can be defined; they are ANDed.
Despite the constraints, the resolution process can return more than one implementation, and more than one instance. If the dependency is multiple, all these instances are solutions. However, for a simple dependency, only one instance must be selected: which one ? The preference clause gives a number of hints to find the "best" implementation and instance to select. The algorithm used for interpreting the preference clauses is as follows: Suppose that the preference has n clauses, and the set of candidates contains m candidates. Suppose that the first preference selects m’ candidates (among them m). If m’=1, it is the selected candidate; if m’=0 the preference is ignored, otherwise repeat with the following preference and the m’ candidates. At the end, if more than one candidate remains, one of them is selected arbitrarily.
A component (instance) is always located inside a composite (instance). The composite may have a global view of its components, on the context in which it executes, and on the real purpose of its components. Therefore, a composite can modify and refine the strategy defined by its components; and most notably the dynamic behavior.
For example, if composite S1Compo wants to adapt the dynamic behavior of all the dependencies from its components and towards components which name matches the pattern A*-lib, it can define a generic dependency like:
<composite name="S1Compo">
...
<contentMngt>
<dependency specification="A*-lib" id="genDep"
eager="true" hide="true" exception="...CompositeDependencyException"/>Suppose a component S1X pertaining to S1Compo has defined the following dependency:
<dependency specification="Acomponent-lib" id="S1XDep" fail="exception"/>When an instance inst of S1X will try to resolve dependency S1XDep, since Acomponent-lib matches the pattern A*-lib, the generic dependency overrides the S1X dependency flags (fail and exception) and extends S1XDep with the eager and hide flags.
eager="true" means that the S1XDep dependencies must be resolved as soon as an instance of S1X is created. By default, eager="false", and dependencies are resolved at the first use of the associated variables in the code.
fail="exception" means that, if the S1XDep dependency fails, APAM will throw the exception ResolutionException on the thread that was trying the resolution. This value overrides the fail definition set on S1XDep.
hide="true" means that, if the S1XDep dependency fails, all the S1X instances are deleted, and the S1X implementation is marked invisible as long as the dependency S1XDep cannot be resolved.
Invisible means that S1X will not be the solution of a resolution, and no new instance of S1X can be created. All S1X existing instances being deleted, the actual clients of S1X instances, at the next use of the dependency, will be resolved against another implementation and another instance. But if a thread was inside an instance inst of S1X at the time its dependency is removed, the thread continues its execution, until it leaves inst normally, or it makes an exception. No other thread can enter inst since it has been removed.
If the hide flag is set, it overrides the component wait flag because the instance will be deleted. But hide and exception are independent which means that in case of a failed resolution, the client component can both receive an exception and be hidden (but cannot wait if hidden). If there is no composite information, only the component "fail" policy applies. If both exceptions are defined, only the composite one is thrown.
exception=”Exception class” means that, if the S1XDep dependency fails, APAM will throw the exception mentioned in genDep (the full name of its class) on the thread that was trying the resolution. This value overrides the exception value set on S1XDep.
This ensures that the current thread which is inside the instance to hide has to leave that instance, and that no thread can be blocked inside an invisible instance.
Important notes: The hide strategy produces a failure backward propagation. For example, if S1XDep fails, APAM hides component S1X and deletes all the inst incoming wires. If an instance y of component Y had a dependency toward inst, this dependency is now deleted. At the next use of the y deleted dependency, since S1X cannot longer be a solution (it is hidden), APAM will look for another component satisfying the Y dependency constraints. If this resolution fails (no other solution exist at that time), and if the Y dependency is also "hide", y is deleted and Y is hidden. The failure propagates backward until a component finds an alternative solution.
This had two consequences: first, it ensures that the application is capable to find alternative solutions not only locally but for complete branches (for which all the dependencies are in the hidden mode). Second, the components are fully unaware of the hidden strategy; the strategy is per composite, which means this is only contextual; it is an architect decision, not an implementer one.
Generic dependencies can express generic constraints:
<composite name="S1Compo">
...
<contentMngt>
<dependency specification="A*-lib">
<constraints>
<instance filter="(OS=Linux)" />
</constraints>
</specification>
</contentMngt>In the example, all the components trying to resolve a dependency toward instances of specifications matching A*-lib will have the associated properties and constraints.
The constraints that are indicated are added to the set of constraint, and appended to the list of preferences, for all the resolutions involving the matching components as target.
In the example, all instances of specifications matching A*-lib must match the constraint OS=Linux. Note that it is not possible to check statically the constraint, since the exact target specification is unknown, and therefore we do not know which properties are defined. If a property, in a filter, is undefined, the filter is ignored. For example, if an instance does not have the OS property, the filter containing the expression (OS=Linux) is ignored.
In APAM, with respect to the platform, a composite (implementation or instance) can export its components (implementations or instances), or import components exported by other composites. This control is performed during the dependency resolution. A dependency from an instance client c in composite cc toward a provider instance p of implementation P is valid (i.e. a wire will be created from c to p) if :
- 1. visible(c, p) ^ import(cc, p);
- 2. visible(c, P) ^ import(cc, P) ^ instantiable(P).
Expression is either a Boolean ("true" or "false") or an LDAP filter to be applied to the component candidates.
A composite designer must be able to decide whether or not to import the instances exported by other composites. This is indicated by the tag <import implementation="Expression"> or <import instance="Expression">. If the target implementation or instance matches the expression, the platform must try to import it if possible. By default, the expression is "true", i.e., the composite first tries to use whatever is available in the platform.
<import implementation="(b=xyz)" instance="false"/> <!--default is true -->import(cc, p) is true if, in composite cc, component p matches the corresponding expression (implementation if p is an implementation, instance otherwise).
In this example, the current composite cc will try to import the implementations that match the expression (b=xyz), but never an instance (instance="false").
If we have <import implementation="false" instance="false"/>, the composite will have to deploy all its own implementations from its own repositories, and create all its instances. It means that it is auto-contained and fully independent from the other composites and components. It can be safely (re)used in any application. Nevertheless, its resolution constraints can include contextual properties such that it can adapt itself to moving context, still being independent from its users.
visible(x, y) is always true if x and y are in the same composite.
If no export tag is present, visible(x, y) is true.
If an export clause is present, only those components matching the export clause can be visible:
<export implementation="Exp" instance="Exp"/> <!-- true by default -->
<exportApp instance= "Exp" /> export means that the components contained in the current composite matching the expression are exported toward all the composites. An implementation can be inside more than one composite type with different export tags; the effective export if the most permissive one. export(x) is true by default.
For example, <export implementation="false" instance="false"/> means that the composite is a black box which hides its content; it does not share any of its service with other composite (except if exportApp allows some services to be visible inside the current application).
exportApp means that the instances contained in the current composite and matching the expression can be imported by any composite pertaining to the same application. exportApp(x) is false by default.
For example, <export instance="false"/><exportApp instance="true"/> means that the services the current composite instance contains are visible only inside the current application.
An instance pertains to a single composite instance; therefore the instances in a platform are organized as a forest. An application is defined as a tree in that forest (i.e., a root composite instance). Therefore, two composite instances pertain to the same application if they pertain to the same instance tree.
By default (none of the above tags are present) a composite exports everything it contains, and imports everything available.
In summary, visible(x, y) = true if one of the following expressions is true:
- composite(x) = composite(y) or
- export(y) = true or // true if no export tag
- (exportApp(y) = true) ^ (app(x) = app(y)) // false if no exportApp tag
A composite type is an implementation, and as such it can indicate its dependencies, as for example:
<composite name="S1Compo" mainImplem="S1Main" specification="S1">
<dependency specification="S2" multiple="true" id="S2Many">
<constraints>
<implementation filter="(apam-composite=true)" />
<instance filter="(scope=global)" />
</constraints>
</dependency>
<dependency interface="fr.imag.adele.apam.test.s2.S2" id="S2Single">
<preferences>
<implementation filter="(x >= 10)" />
</preferences>
</dependency>This definition says that composite S1Compo has a dependency called S2Many towards instances of specification S2; multiple="true" means that each instance of S1Compo must be wired with all the instances implementing S2 and satisfying the constraints. When an instance of S1Compo will have to resolve that dependency, first APAM selects all the S2 implementations satisfying the constraint (apam-composite=true), and then APAM selects, all the instances of these implementations satisfying the constraint (scope=global).
The dependency called S2Single is toward an interface. When it has to be resolved, APAM looks for an implementation that implements that interface, and preferably one instance satisfying (x >= 10), any other one otherwise. A single instance of that implementation will be selected and wired.
Suppose that an instance A-0 of implementation A is inside an instance S1Compo-0 of composite S1Compo. Suppose that implementation A is defined as follows:
<implementation name="A" classname="...A" specification="SX">
<dependency interface="...I2" multiple="true" field="linux" id="toLinux">
<constraints>
<implementation filter="(OS=Linux)" />
</constraints>
</dependency>
<dependency specification="S2" field="s2" id="fastS2">
<preferences>
<implementation filter="(speed > 15)" />
</preferences>
</dependency>Finally, suppose that specification S2 provides interfaces I1 and I2:
<specification name="S2" interfaces="...I1, ...I2" >
<definition name="OS" type="Windows, Linux, Android, IOS" />
<definition name="speed" type="int" />When instance A_0 uses for the first time its variable linux, APAM checks if the A_0 dependency toLinux is a dependency of its embedding composite. Indeed, I2 is part of specification S2, and matches both dependencies 2Many and S2Single defined in S1Compo. However, toLinux being a multiple dependency, only S2Many can match the dependency, and therefore, APAM considers that toLinux has to be promoted as the S2Many dependency.
Because of this promotion, APAM has to resolve S2Many that will be associated with a set of S2 instances matching the s2Many constraints (if any); then the same set of instances will be considered for the resolution of toLinux, therefore a sub-set (possibly empty) of s2Many instances will be solution of the toLinux dependency.
The fastS2 dependency, being a simple dependency will be resolved either as as the S2Single instance, or as one of the targets of S2Many.
If, for any reason, an internal dependency is a promotion that cannot be satisfied by the composite, the dependency fails i.e. APAM will not try to resolve the dependency inside the composite.
A composite can explicitly, and statically, associate an internal dependency with an external one. For example, composite S1Compo can indicate
<promote implementation="A" dependency="fastS2" to="S2Single" />
<promote implementation="A" dependency="toLinux" to="S2Multi" />It means that the dependency fastS2 of A is promoted as the dependency S2Single of S1Compo; in which case the constraints of fastS2 are added to the list of the S2Single dependency. It is possible to build, that way, static architectures as found in component models; however this is discouraged since it requires a static knowledge of the implementations that will be part of a composite, prohibiting opportunism and dynamic substitution.
