Thought Experiment - Elements and Components

In this chapter, we will delve into the world of Eight, understanding its processes of coding, combining, configuring, running, and adjusting, gradually clarifying its unique worldview and methodology. Similarly, we will start with some simple examples to illustrate these differences.

What is an Element

At first glance, Eight’s code can be perplexing. They appear headless and tailless, with no clear beginning or end. Furthermore, they seem unrelated to each other, with hardly any class referencing another, making it difficult to trace the thought process. Additionally, each class is written in a very abstract manner, making it unclear what they are meant to do. For example, consider the following:

public class OverflowProcessor<T, R> implements IProcessor<T, R> {
	protected IProcessor<T, R> proxy;
	protected Integer max;
	
	public void setMax(Integer max) {
		if (max != null && max > 0) this.max = max;
	}

	public void setProxy(IProcessor<T, R> proxy) {
		this.proxy = proxy;
	}

	@Override
	public R process(T in) {
		if (max != null && in != null) if (in.getClass().isArray() && Array.getLength(in) > max || in instanceof String && ((String)in).length() > max 
				|| in instanceof Map && ((Map) in).size() > max || in instanceof Collection && ((Collection) in).size() > max) throw new PlatformException(PlatformError.ERROR_PERFORMING_FAIL, "Data overflow. The maxLength is " + max);
		return proxy.process(in);
	}
}

Yet, such code forms the ever-changing system of Eight. These are elements, the most basic particles in the world of Eight, the atoms that make up various modules.

What is this code for? Why is it written this way? This brings us back to our previous topic: interfaces are the connections between substances, but what are substances?

In Eight’s worldview, substances are wholes composed of smaller substances connected together. Not only do the connections between substances have basic forms, but the substances themselves are also composed of basic particles, albeit in such large numbers that they are difficult to enumerate. These basic particles are also substances, but because they are more fundamental, they are more abstract and harder to understand; because they are more fundamental, they are more universal and can be used in various places; and because they are more fundamental, they must be combined to become concrete, to become a describable “substance” or “module.” Indeed, this is the full picture of eight: things themselves are composed of things and connections. These basic particles that make up things are elements. Element

The code above describes a common process: when the input parameter (adapted to several basic types) exceeds a set threshold, it exits abnormally; if it meets the set criteria, it is handed over to the proxy process for further execution. This code is often embedded in a segment of processing logic for validity checks. Here is a hint, the similar-concept agreed upon by Eight is essentially a way of decomposing substances, dividing things into neatly cut parts, any of which can connect with another part of corresponding interface to produce different effects. The dynamism mentioned in the previous chapter is just one feature. The embedding here modifies the processing logic of the substance itself. Imagine if our Search module had a functional adjustment, refusing to process queries with field lengths exceeding 1KB. We wouldn’t need to rewrite the code; we could simply place the OverflowProcessor in front of Search module. Also, note the setMax method: all set methods in Eight can be parameterized and dynamically injected at runtime to take effect immediately. So, if we need to change the threshold to 10KB at some point, it would be an instant matter, perhaps taking a few nanoseconds.

The OverflowProcessor is a typical element, fully embodying the characteristics of Eight’s coding. Almost all of Eight’s code is like this:

Necessary Constraints
- For any element, for undefined references and operations (including but not limited to objects, methods, or processes) not contained within this element, one or more interfaces and methods defined by the similar-concepts interfaces should be used to describe;
- For any element, if providing services and operations (including but not limited to objects, methods, or processes) to the outside, these services and operations should be provided by implementing one or more interfaces and methods defined by the similar-concepts interfaces;
Optional Constraints
- Unless there is a clear and stable dependency relationship, elements are not recommended to inherit or reference other types;
- If an element’s attribute fields need adjustment, they should be provided via set methods;

Here is an explanation of these constraints. The first constraint means coding based on self, aiming for domain autonomy. The term undefined has a primary meaning of external associations being undefined when writing code. A common practice is to agree on interfaces with the service provider (of course, everyone knows that dependencies arise at this time), but Eight does not recommend this approach. Eight suggests that developers focus on the module itself, rationally analyzing the scope of current domain problems, applying the similar-concepts model to hypothesize external associations within the scope, and then using code to describe external things. Eight does not recommend excessive and detailed negotiations and communications. If necessary, negotiations and communications should remain at the level of module domain division. Too deep communication can lead to mutual intrusion of developers’ thoughts, causing implicit dependencies, which is a cause of system instability.

There might be a question here: what if a lack of understanding leads to an inability to connect? In principle, if both modules are complete within their domain, meaning they have thought of and implemented the necessary functions, then any connection issues should be related to connections. Such problems can be easily solved using the adaptation techniques mentioned later. If there is a functional mismatch, at least one side must have deviated in their domain implementation, which is not caused by communication but by a lack of sufficient understanding of the business domain (think of Spinoza). To determine which module has the problem, usually, reviewing the module and business will suffice (after all, most developers share similar-concepts). It can also be approached empirically by looking at how these modules integrate with other environments, though this may not always be accurate or suitable.

The second meaning is simple: even if the external module’s approach is known, the similar-concepts interfaces should still be used. The reason is clear: the other party is also constrained and provides the similar-concepts interfaces externally. This ensures that there is no dependency relationship in form, allowing one side’s modifications and adjustments at runtime not to affect the other.

The third meaning is more profound: what counts as undefined? Is it currently undefined or potentially undefined in the future? Eight’s suggestion is that change is undefined. Any part that can change is undefined in the current domain and should be described.

For references to the undefined part, one or more of the similar-concepts interfaces must be used. These references are often injected as properties via set methods.

The second constraint is straightforward—follow the rules. Since this world requires us to connect in this manner, we must conform. What value does our domain have to the outside world? What function does it serve? It must be presented in a way that the outside world can understand.

The third optional constraint aims to avoid forming a web of dependencies. This is not absolute, of course. If the domain itself has stable and necessary external dependencies, such as an element responsible for redis calls, integrating redis utility classes is appropriate. The fourth constraint is a requirement of Spring’s IoC, and Spring’s role is consistent with Eight’s, used for decoupling. Later, we will see that Spring’s position in the Eight system is no less than OSGi.

Combining the above constraints, it is evident that Eight’s design goal is to force developers to create independent, cohesive, and universal modules, producing valuable and widely used modules. Reducing communication is actually a side effect (although managers may like to hear this, as it objectively brings greater development decoupling. But this is not Eight’s main goal; as long as module independence is ensured, deep communication is not necessarily bad). If a module is universal, it must have minimal external perception, as we cannot communicate with every user to decide the internal logic of the module.

Take the OverflowProcessor as an example, from its own perspective: in my world, I am a proxy reviewer. I know I proxy another IProcessor, but I don’t know or care who it is or what it does. My way of connecting with the world is that I am also an IProcessor, so various Objects will come through process(Object in). I don’t know where they come from or what they are, and I don’t care! But if they are of the types I can handle, I have to work. I need to determine if they meet the rules. What are the rules? They must not exceed max. What is max? I don’t know, and I don’t care! Someone will tell me when the time comes. If they meet the rules, I pass them to that… who cares! If they don’t, I throw an exception and discard them. I am me; I know nothing about the outside world, but I can do what I need to do.

Composition of Components

Eight provides dozens of packages with hundreds of elements. Elements are still small units and do not yet form meaningful wholes. We need to assemble them. Thanks to the community, we have a natural assembly tool: Spring. Its design is fantastic. As you delve deeper, you will find that Eight has very high assembly requirements, with many unusual configuration methods, but nothing that cannot be achieved through Spring. It is truly a magical masterpiece.

Here is a small example, showing a configuration segment, a complete module.

<bean class="net.yeeyaa.eight.dispatch.Dispatcher" name="preprocessor">
	<property name="pattern" value="([\w.$]+)@"/>	
	<property name="dispatcher" ref="callback"/>
	<property name="cache" value="true"/>
</bean>

<bean class="net.yeeyaa.eight.core.processor.CallbackProcessor" id="callback">
    <property name="processor" ref="beanHolder"/>
    <property name="paras" value="next"/>
</bean>

<bean factory-bean="callback" factory-method="getValue" name="nextname" scope="prototype"/>

<bean class="net.yeeyaa.eight.core.processor.UniversalProxy" id="next" scope="prototype">
	<property name="invoker">
		<bean class="net.yeeyaa.eight.osgi.runtime.ProxyBean">
			<property name="invoker" ref="invoker"/>
			<property name="name" ref="nextname"/>
		</bean> 		
	</property>
</bean>

The key classes are as follows, both very brief:

public class CallbackProcessor implements IProcessor<Object, Object> {
	protected ThreadLocal<Object> local = new ThreadLocal<Object>();
	protected IProcessor<Object, Object> processor;
	protected Object paras;
	protected IProcessor<Object, Object> valuePrcoessor;

	public void setProcessor(IProcessor<Object, Object> processor) {
		this.processor = processor;
	}

	public void setValuePrcoessor(IProcessor<Object, Object> valuePrcoessor) {
		this.valuePrcoessor = valuePrcoessor;
	}

	public void setParas(Object paras) {
		this.paras = paras;
	}

	public Object getValue() {
		if (valuePrcoessor == null) return local.get();
		else return valuePrcoessor.process(local.get());
	}

	@Override
	public Object process(Object paras) {
		this.local.set(paras);
		Object ret = processor.process(this.paras); 
		this.local.remove(); 
		return ret;
	}
}

public class Dispatcher implements IProcessor<Object, Object>{
	protected IProcessor<String, IProcessor<Object, Object>> dispatcher;
	protected Boolean cache = false;
	protected Pattern pattern;
	protected ConcurrentHashMap<String, IProcessor<Object, Object>> processors;
	protected String error;
	
	public void setError(String error) {
		this.error = error;
	}

	public void setDispatcher(IProcessor<String, IProcessor<Object, Object>> dispatcher) {
		this.dispatcher = dispatcher;
	}

	public void setCache(Boolean cache) {
		if(Boolean.TRUE.equals(cache)) {
			processors = new ConcurrentHashMap<String, IProcessor<Object, Object>>();
			this.cache = cache;
		}
	}

	public void setPattern(String pattern) {
		if(pattern != null) this.pattern = Pattern.compile(pattern);
	}

	@Override
	public Object process(Object msg) {	
		if(msg != null){
			String key;
			if (msg instanceof ServiceMsg) key = ((ServiceMsg) msg).name;
			else key = new JSONObject(msg.toString()).getJSONObject("msg").getString("name");
			Matcher matcher = pattern.matcher(key);
			if(matcher.find()){
				String name = matcher.group(1);
				IProcessor<Object, Object> processor = null;
				if(cache) processor = processors.get(name);
				if(processor == null) processor = dispatcher.process(name);
				if(processor != null){
					msg = processor.process(msg);
					if(cache) processors.put(name, processor);
				}else if(error == null) throw new PlatformException(PlatformError.ERROR_PARAMETERS);
				else return error;
			}else if(error == null) throw new PlatformException(PlatformError.ERROR_PARAMETERS);
			else return error;
		}
		return msg;
	}
}

To explain, beanHolder can be understood as beanFactory, which allows access to beans in the current springContext. The ProxyBean segment is a general configuration that will be frequently encountered, as it is the connection point to the external world, interfacing with a linker. Our module connects with unknown external modules through it. So it masquerades as IUniversal (UniversalProxy), meaning it can be any interface in the Eight world. But this is not important; it is a minor role in this context. The only thing to note is that it determines which external linker to call based on the name attribute.

All the information is provided. Now, does anyone know what this module does? Why is it configured this way? Where is the runtime entry point? What processing is done to achieve what result? How is it done? If you are interested, feel free to leave me a message~~

Eight has many more such intriguing configuration methods.

Spring also assists Eight in parameterizing module configurations, allowing Eight’s module parameters to be dynamically adjusted at runtime. This is the most common configuration method, for example:

<bean class="net.yeeyaa.eight.osgi.runtime.BundleHttpService" destroy-method="destroy" id="httpService" init-method="initialize">
    <constructor-arg ref="log"/>
    <property name="context" ref="context"/>
    <property name="services"> 
	   <list>	
		   <bean class="net.yeeyaa.eight.osgi.runtime.BundleHttpService$ServiceInfo">
		   		<property name="alias" value="${framework.test.index.alias:/index}"/>
		   		<property name="name" value="/html"/>			
		   </bean>				   	   		   
	   </list>		   
    </property>  	
</bean>

Here, an alias configuration is parameterized as ${framework.test.index.alias:/index}, meaning the default value is ‘/index’. This class is used to publish HTTP services, with alias representing the URL corresponding to the service. By modifying the parameter, we can dynamically move the service to any URL at runtime.

<bean class="net.yeeyaa.eight.access.http.HttpServer" id="httpServer">
    <constructor-arg ref="log"/>
    <property name="https" value="false"/>
	<property name="executor" ref="pool"/>
	<property name="port" value="${framework.http.httpServer.port:7241}"/>
	<property name="keyfile">
		<bean class="net.yeeyaa.eight.common.storage.MockStorage">
            <property name="resource">
                <value>classpath:META-INF/config/http.key</value>
            </property>
        </bean>
	</property>
	<property name="password" value="simulator"/>    	
</bean>

This is another example, straightforwardly showing that framework.http.httpServer.port is used to set the port number. We can adjust the service’s bound port number at runtime.

In fact, undefined things are not limited to the connection between self and the outside world; they also include the state of self at runtime. These belong to self but are variable and need to be externalized. In the Eight framework, you can delegate any undefined state to be set at runtime, and Eight ensures they can be changed at runtime.

The above parameter adjustments require reloading a springContext (note that reloading the bundle or Classloader is not necessary), generally taking sub-millisecond. For re-publishing modified classes, it takes longer, generally a few milliseconds to tens of milliseconds depending on module size (after all, it only involves locally loading new classes and creating services). Meanwhile, the old module will maintain service during the new module’s loading process, and will be released and recycled after loading.

Is there a faster way to change? If the parameter does not require service reconfiguration (e.g., the max parameter mentioned earlier, which is only related to business, is suitable, whereas framework.http.httpServer.port, which binds the port, requires corresponding service reinitialization, so it does not meet the requirement), it can be exposed as a JMX parameter, such as the aforementioned formatter:

<bean class="org.springframework.jmx.export.MBeanExporter" id="exporter" lazy-init="false">
	<property name="beans">
		<map>
			<entry key="bean:name=${framework.test.formatter:framework.test.formatter}" value-ref="formatter"/>
		</map>
	</property>
	<property name="assembler">
		<bean class="org.springframework.jmx.export.assembler.MethodNameBasedMBeanInfoAssembler">
			<property name="managedMethods">
				<value>setMode</value>
			</property>
		</bean>
	</property>
</bean>	 
<bean class="net.yeeyaa.test.format.Formatter" id="formatter">    
	<property name="processor">
		<bean class="net.yeeyaa.eight.core.processor.UniversalProxy">
			<property name="invoker">
				<bean class="net.yeeyaa.eight.osgi.runtime.ProxyBean$$Proxy">
					<property name="invoker" ref="invoker"/>
					<property name="name" value="${framework.test.formatter.processor:processor}"/>
				</bean> 		
			</property>
		</bean> 
	</property> 		   	  
</bean>

Invoking the JMX interface for setMode is essentially a local call, consuming merely a few clock cycles without necessitating the reloading of any components. This can all be configured through binding, eliminating the need for coding decisions at the time of implementation. At most, the coding phase may require handling interfaces for undefined scenarios. Therefore, it is advisable to provide set methods for properties as much as possible, as you never know when a property might need to be altered.

Elements are integrated into a whole using Spring, which then acquires a clear business significance within its domain: this substance is termed a component. A component is the basic unit that can be dynamically loaded, updated, uninstalled, and configured with parameters during runtime. The interaction and integration between components utilize the OSGi (Felix iPojo) framework. Introducing Felix and iPojo in detail would be a substantial task, and those interested can refer to iPojo. I will not delve into it further here.

We use Spring to connect a group of elements into a block, which is then packaged into a bundle using Felix; this constitutes the physical form of a component.

Module Boundaries

At this point, you might wonder: why do we use Spring for assembly at one moment and OSGi at another, creating such a complex two-tier structure? Isn’t the relationship between components fixed once they are assembled with Spring? How can we maintain dynamism? If it is not used to respond to changes, what is its value? This touches on a methodological dilemma: what exactly is an substance? What is a connection? What is the standard? If these cannot be distinguished, the boundaries of a module cannot be determined.

We previously discussed how so-called substances are composed of smaller substances interconnected, explored layer by layer down to the basic granules—elemetns. Elements are certainly indivisible; are they the boundaries? No, they lack significance and do not constitute substances in principle, merely representing a local part. Practically, it is unimaginable for an enormous number of elements to operate as individual modules. So, what level constitutes the boundary of a module?

Let’s take ourselves as an example. Are your relationships with those around you stable? If they are not stable enough, could a friend you cooperate with today be a stranger tomorrow? If a module solidifies the relationships between you, it would need to be rewritten. Does this mean that as long as there are two independent substances, they must be separated into independent modules? Not necessarily; think of the story of Boya and Ziqi, where people can share joy and sorrow. Conversely, are individual substances necessarily stable? What if someone suddenly turns into another person at midnight? Sounds a bit scary, doesn’t it? Cinderella, the poor woman, clearly had no stable self; her essence was bestowed by external adornments. If she believed the prince loved her for who she was, why would she flee?

By now, you should understand my point. Empiricists do not believe in a clear dividing line for module boundaries, but there is a clear standard for module division: change. If stable connections can be solidified into a module, the number of modules in our system will be fewer, often meaning easier management and maintenance. Conversely, unstable connections need to be externalized, reducing the cost of change, which is also conducive to management, maintenance, and development. However, stability and change vary from environment to environment, even differing across periods. Stability is relative; the only constant is change itself. Module boundaries can only be determined in practice, which is the empiricist attitude.

Thus, you should understand why there are two assembly frameworks, Spring and OSGi. Eight’s worldview does not distinguish module division points; it is consistent from bottom to top, with basic granules interconnected through similar-concepts interfaces, forming an integrated system layer by layer. Whether these connections are stable is beyond its scope. In practice, the division between Spring and OSGi lies at the points of change, i.e., the boundaries of modules. Internal connections rely on Spring, while external connections rely on OSGi, but this boundary is not so definite. Spring is relatively fixed, but if change unfortunately occurs within a module, necessitating further division to separate unstable connections, Spring’s configuration-based assembly can easily separate at the division point, with consistent connection methods (similar-concepts interfaces). Conversely, if practice proves that the relationship between two modules is very stable, they can be merged into a single module for a simpler system structure and easier maintenance and management, with Spring naturally taking on the assembly task. Thus, using Spring is a compromise between change and solidification, with the goal still being to respond quickly to change.

This brings us to why Eight provides its elements library in an uncommon manner. Indeed, the core purpose is to separate generality from contingency, enabling the system to respond quickly to change. Who understands this?

To be specific, it is still about providing basic support for modular development. Eight aims to surpass all previous frameworks by offering a highly dynamic system, facing many challenges under current technological conditions, one of which is developer needs. Eight must offer developers an easy implementation method while preventing the unconscious formation of dependency networks, necessitating careful design of the elements library.

This elements library must achieve two goals: first, sufficiency (relatively speaking), meaning that once developers enter the Eight environment, they do not need to add additional packages to the Eight base; the dependencies provided by the environment are sufficient. This prevents various packages from being introduced into the foundational environment, complicating the dependency network and destabilizing the system. Additionally, package and version management in an OSGi environment is a headache, so it is best not to burden users with it, especially since OSGi packages have become hard to find in recent years. Another important aspect is that Eight will be distributed as a foundational platform to large-scale clusters, and if the runtime environment keeps changing, with each project being different, usability will be greatly reduced. Using containers like k8s is fine, but deploying on a rudimentary cluster is problematic.

Of course, this issue is not too critical. After all, OSGi bundles can dynamically load libraries, bringing along the necessary components within the module, without needing to stuff anything into the platform environment. So, even if Eight does not cover all libs needs, the goal can still be achieved. However, this approach makes modules heavier, increasing the loading process cost, and another drawback is the difficulty of intercommunication between modules. Different bundles use different Classloaders, and even if the same class (in a binary sense) is loaded, they do not recognize each other, causing type exceptions if data transfer occurs (not recommended).

Therefore, it is necessary for Eight to provide a library encapsulating common libs. After all, when a module enters the environment, the only three dependencies are: 1) the Java core library; 2) similar-concepts interfaces; 3) the Eight elements library. Of course, some common libraries used by Eight will also be accessible in the base, but it is generally not recommended for modules to reference them directly, as these libraries might change.

This explains why Eight’s components are abstract and independent: 1) they have diverse uses. For instance, some are used for search, while others assume you have filtered the results, requiring flexible combinations to meet these needs; 2) they must remain stable and unchanged. This is easy to understand; if Eight’s elements library changes, it is not just a cluster deployment issue: it could be embedded in various places for different purposes. Hence, it requires highly abstract simplicity, with any changes considering internal consistency.

Secondly, Eight’s elements library usage differs from typical libs; Eight is for building frameworks. Typically, using a lib involves creating an object, requiring code to recognize the lib, which creates dependencies. Eight is different; likening a module to a house, Eight’s elements library aims to erect the framework, with walls, doors, windows, floors, and ceilings in place, just awaiting furniture. The difference is that one requires you to build the house, while the other makes you the furniture. Once the framework is built, the business module’s recognition of the environment is greatly reduced, often holding a few IProcessor or IResource objects without caring where they come from, just using them as needed. This leads to an omnipotent self development state, using whatever is needed. Eight becomes the world that meets all needs. To meet these needs with a limited library, high abstraction is essential. Spring’s various uncommon uses also reflect this. Thus, for Eight, configuration is an art.

There are many benefits. Besides obtaining a highly dynamic system during development, and the ability to cut within-module changes, modules can operate lightly, significantly improving efficiency. With a few hundred classes in Eight, frequently used ones are repeatedly utilized to accomplish tasks previously handled by large frameworks, often being instantiated dozens or hundreds of times, likely making every piece of code a hotspot. Configuration is not as troublesome as imagined; compared to code, it is much easier to copy-paste, and once familiar, it becomes highly standardized. Once the framework is set, only minor additions are needed.

Ultimately, you do not have to develop according to the above principles. You can entirely discard those abstract elements and incomprehensible configurations; Eight will not force you to write code in this manner: this is the elements library’s task (though you might want to build more elements libraries suited to your business, which is not forbidden). You can write a complete module in your usual way, with closely related classes calling each other in the code, and bring along a bunch of libs in the bundle to form a self-contained entity. You substance need to ensure:

For undefined references and operations (including but not limited to objects, methods, or processes) not contained within the module, use one or more interfaces and methods defined by the similar-concepts interfaces to describe them;
For services and operations provided by the module externally (including but not limited to objects, methods, or processes), implement one or more interfaces and methods defined by the similar-concepts interfaces;
Most importantly, do not cross-reference and depend on other modules, and do not throw libs into the platform environment.

Then you can run as a module in Eight, intercommunicating with other modules without mutual dependencies. Their changes will not affect you, nor will you affect them. However, whether your module is easy to parameterize, reload, cut, or respond to unknown changes is another matter.

Finally, consider an experiment in thought: what should the form of Search and UserInterface be in Eight? When the Search service needs to add a dir query parameter, how should this change be responded to?

PREVIOUSEight's Starting Point - Similar-Concepts Interfaces

NEXTEver and Everwhere - Connections, Operations, and Constant Change