docs: add "Mixture of continuous and discrete"

docs/writings/4.1 Interface of continuous and discrete.md

@@ -8,7 +8,11 @@
 If you remove one grain of sand from a heap of sand, it is still a heap. If you keep removing grains of sand, eventually you will have only one grain of sand left. Is it still a heap?
 :::
 
-The key point of this paradox lies in the continuous nature of the concept - `heap`. The concept `heap`, is actually never well-defined when we use it as natural language. It also does not have a formal definition in any way. You knowledge about the heap might vary infinitesimally. For a same object, you might think that is a heap today and not a heap tomorrow.  
+The key point of this paradox lies in the continuous nature of the concept - `heap`. The concept `heap`, is actually never well-defined when we use it as natural language. It also does not have a formal definition in any way. You knowledge about the heap might vary infinitesimally.
+
+::: tip Observation
+For a same object, you might think that is a heap this second and not a heap in the next second. However, you do not think your knowledge about the heap is changing even you give different answer. This proves that the concept `heap` is continuous because infinitesimal variation does not matter. 
+:::
 
 ### Attack to any continuous concept
 
@@ -29,11 +33,34 @@ For example, you can attack the concept `machine learning` by the following way:
 - You can adjust the process by replacing all the learning steps with human-made step except for one machine-made noise step. It is ridiculous if there are 100000 human-made steps and 1 useless machine-made step. However, it is still an instance of `machine learning`.
 :::
 
-## Topology
+Importantly, nearly all the big concepts are continuous, including `philosophy`, `science`, `understanding`, `knowledge`, `freedom`, `democracy`, etc. They are all vulnerable to this attack. Though they are useful concepts, people should keep in mind that they can never have a clear definition. This impossibility of this has been proved by the history.
 
-Topology is an example where continuous entities can be unambiguously represented by discrete entities.
+## Mixture of continuous and discrete
+
+There are knowledge that is neither continuous nor discrete. They are mixture of both. For example, though I claimed that neural networks mainly carry continuous knowledge, they also carry discrete knowledge by its network structure. The network structure is discrete and the weights are continuous.
+
+The tree of knowledge in EvoNote is the same. The tree structure is discrete and the knowledge in the nodes are continuous because they are natural language.
+
+### Machine learning engineers 
+
+Machine learning engineers does a funny job. They design the discrete structure of the neural network and train the continuous weights. In this way, they mix the discrete and continuous knowledge together. 
+
+In this process, one interesting point is the roles of human and machine. The discrete structure is designed by human and the continuous weights are trained by machine. Therefore, machine only deals with continuous knowledge and the discrete knowledge are handled by human. This matches the performance of the model as products - they are good at continuous tasks and bad at discrete tasks.
 
-Condense matter physics loves topology very much. Part of the reason is the topological properties of condensed matter system nearly the only way to describe them in a discrete way.
+> Network Architecture Search (NAS) is a process that tries to automate the design of the discrete structure. However, it failed heavily in the competition with the transformer structure. This proves that machine learning are not good at discrete tasks again.
 
-## Natural language classifier
+## Math that bridges continuous and discrete
+
+Math provides some concrete bridge between continuous and discrete. This kind of bridge is hard to find from daily life knowledge. This makes math beautiful and holy.
+
+### Group theory
+
+The space where the continuous knowledge lives might have a symmetry described by a certain Lie group. Group theory offers a way to analyze these continuous knowledge by analyzing its Lie group. For example, the Lie group might have a countable number of generators, which gives a discrete way to analyze the continuous knowledge. We can also analyze the representation of the Lie group, which will make the representation of it more discrete if we can decompose it into irreducible representations.
+
+Using the discrete knowledge found by group theory have been applied to design neural networks. [Equivariant neural networks](https://arxiv.org/abs/2006.10503) are one example. 
+
+### Topology
+
+Topology is an example where continuous entities can be unambiguously represented by discrete entities.
 
+Condense matter physics loves topology very much. Part of the reason is the topological properties of condensed matter system nearly the only way to describe them in a discrete way.