**Preprints
& selected publications:**

2020 __2019 ____2018 __2017 2016 2015 2014 2013 2012 2011 2010 2009 2008
2007 2006 2005
2004 2003 2002
2001 2000 1999
1998 1997 1996
1995 1994 1993
1992 1991 1980-1990

AN Gorban, VA Makarov,
IY Tyukin,** ****High-Dimensional
Brain in a High-Dimensional World: Blessing of Dimensionality**

*Entropy*
22 (1), 82, https://doi.org/10.3390/e22010082

High-dimensional data and high-dimensional representations of reality are
inherent features of modern Artificial Intelligence systems and applications of
machine learning. The well-known phenomenon of the “curse of dimensionality”
states: many problems become exponentially difficult in high dimensions.
Recently, the other side of the coin, the “blessing of dimensionality”, has
attracted much attention. It turns out that generic high-dimensional datasets
exhibit fairly simple geometric properties. Thus, there is a fundamental tradeoff between complexity and simplicity in high
dimensional spaces. Here we present a brief explanatory review of recent ideas,
results and hypotheses about the blessing of dimensionality and related
simplifying effects relevant to machine learning and neuroscience.

Inventors: Ilya Romanenko, Alexander Gorban, Ivan Tyukin

**Image
Processing, US Patent 10,489,634, Nov 26,
2019**

Assignee: Apical Ltd & University of Leicester; Priority date: 2017-09-26.

A.N. Gorban, **Universal Lyapunov functions for non-linear reaction networks.** *Communications in Nonlinear Science and
Numerical Simulation*, 2019, 79, 104910.

In 1961, Rényi discovered
a rich family of non-classical Lyapunov functions for
kinetics of the Markov chains, or, what is the same, for the linear kinetic
equations. This family was parameterized by convex functions on the positive
semi-axis. After works of Csiszár and Morimoto, these
functions became widely known as *f*-divergences or the Csiszár–Morimoto divergences. These Lyapunov
functions are universal in the following sense: they depend only on the state
of equilibrium, not on the kinetic parameters themselves.

Despite many years of research, no such
wide family of universal Lyapunov functions has been
found for nonlinear reaction networks. For general non-linear networks with
detailed or complex balance, the classical thermodynamics potentials remain the
only universal Lyapunov functions.

We constructed a rich family of new universal Lyapunov
functions for *any non-linear reaction network* with detailed or complex balance. These functions are
parameterized by compact subsets of the projective space. They are universal in
the same sense: they depend only on the state of equilibrium and on the network
structure, but not on the kinetic parameters themselves.

The main elements and operations in the construction of the new Lyapunov functions are partial equilibria of reactions and
convex envelopes of families of functions.

S.Y. Gordleeva,
Y.A. Lotareva, M.I. Krivonosov,
A.A. Zaikin, M.V. Ivanchenko, A.N. Gorban, **Astrocytes
Organize Associative Memory.**

In *Advances
in Neural Computation, Machine Learning, and Cognitive Research III: Selected
Papers from the XXI International Conference on Neuroinformatics,
October 7–11, 2019, Dolgoprudny, Moscow Region,
Russia, *Springer, Cham, 384-391 (2019, October).

We investigate one aspect of the functional role
played by astrocytes in neuron-astrocyte networks present in the mammal brain.
To highlight the effect of neuron-astrocyte interaction, we consider simplified
networks with bidirectional neuron-astrocyte communication and without any
connections between neurons. We show that the fact, that astrocyte covers
several neurons and a different time scale of calcium events in astrocyte,
alone can lead to the appearance of neural associative memory. Without any
doubt, this mechanism makes the neuron networks more flexible to learning, and,
hence, may contribute to the explanation, why astrocytes have been evolutionary
needed for the development of the mammal brain.

Z. Chen, B. Wang, A.N. Gorban, A.N. **Multivariate
Gaussian and Student-t process regression for multi-output prediction.** *Neural Comput
& Applic* (2019). https://doi.org/10.1007/s00521-019-04687-8

Gaussian process model for vector-valued function
has been shown to be useful for multi-output prediction. The existing method
for this model is to reformulate the matrix-variate Gaussian distribution as a
multivariate normal distribution. Although it is effective in many cases,
reformulation is not always workable and is difficult to apply to other
distributions because not all matrix-variate distributions can be transformed
to respective multivariate distributions, such as the case for matrix-variate
Student-*t* distribution. In this
paper, we propose a unified framework which is used not only to introduce a
novel multivariate Student-*t* process regression model (MV-TPR) for multi-output prediction, but also
to reformulate the multivariate Gaussian process regression (MV-GPR) that
overcomes some limitations of the existing methods. Both MV-GPR and MV-TPR have
closed-form expressions for the marginal likelihoods and predictive
distributions under this unified framework and thus can adopt the same
optimization approaches as used in the conventional GPR. The usefulness of the
proposed methods is illustrated through several simulated and real-data
examples. In particular, we verify empirically that MV-TPR has superiority for
the datasets considered, including air quality prediction and bike rent
prediction. At last, the proposed methods are shown to produce profitable
investment strategies in the stock markets.

A.N. Gorban, E.M. Mirkes, I.Y. Tyukin, **How Deep
Should be the Depth of Convolutional Neural Networks: a Backyard Dog Case
Study.*** Cogn Comput *(2019). https://doi.org/10.1007/s12559-019-09667-7

The work concerns the problem of reducing a pre-trained deep
neuronal network to a smaller network, with just few layers, whilst retaining
the network’s functionality on a given task. In this particular case study, we
are focusing on the networks developed for the purposes of face recognition.
The proposed approach is motivated by the observation that the aim to deliver
the highest accuracy possible in the broadest range of operational conditions,
which many deep neural networks models strive to achieve, may not necessarily
be always needed, desired or even achievable due to the lack of data or
technical constraints. In relation to the face recognition problem, we formulated
an example of such a use case, the ‘backyard dog’ problem. The ‘backyard dog’,
implemented by a lean network, should correctly identify members from a limited
group of individuals, a ‘family’, and should distinguish between them. At the
same time, the network must produce an alarm to an image of an individual who
is not in a member of the family, i.e. a ‘stranger’. To produce such a lean
network, we propose a network shallowing algorithm. The algorithm takes an
existing deep learning model on its input and outputs a shallowed version of
the model. The algorithm is non-iterative and is based on the advanced
supervised principal component analysis. Performance of the algorithm is
assessed in exhaustive numerical experiments. Our experiments revealed that in
the above use case, the ‘backyard dog’ problem, the method is capable of
drastically reducing the depth of deep learning neural networks, albeit at the
cost of mild performance deterioration. In this work, we proposed a simple
non-iterative method for shallowing down pre-trained deep convolutional
networks. The method is generic in the sense that it applies to a broad class
of feed-forward networks, and is based on the advanced supervise principal
component analysis. The method enables generation of families of smaller-size
shallower specialized networks tuned for specific operational conditions and
tasks from a single larger and more universal legacy network.

Alex J Bell, Brody H Foy, Matthew Richardson,
Amisha Singapuri, Evgeny Mirkes, Maarten van den Berge, David Kay, Chris Brightling, Alexander N Gorban, Craig J Galbán,
Salman Siddiqui, **Functional CT imaging for identification of
the spatial determinants of small-airways disease in adults with asthma,** *Journal of Allergy and Clinical Immunology,*
144(1), 83-93.

Background

Asthma is a disease characterized by ventilation heterogeneity (VH). A number
of studies have demonstrated that VH markers derived by using impulse oscillometry (IOS) or multiple-breath washout (MBW) are
associated with key asthmatic patient–related outcome measures and airways hyperresponsiveness. However, the topographical mechanisms
of VH in the lung remain poorly understood.

Objectives

We hypothesized that specific regionalization of
topographical small-airway disease would best account for IOS- and MBW-measured
indices in patients.

Methods

We evaluated the results of paired
expiratory/inspiratory computed tomography in a cohort of asthmatic (n = 41)
and healthy (n = 11) volunteers to understand the determinants of clinical VH
indices commonly reported by using IOS and MBW. Parametric response mapping
(PRM) was used to calculate the functional small-airways disease marker PRMfSAD and Hounsfield unit (HU)–based density changes from
total lung capacity to functional residual capacity (ΔHU); gradients of ΔHU in
gravitationally perpendicular (parallel) inferior-superior (anterior-posterior)
axes were quantified.

Results

The ΔHU gradient in the inferior-superior axis provided the highest level of
discrimination of both acinar VH (measured by using phase 3 slope analysis of
multiple-breath washout data) and resistance at 5 Hz minus resistance at 20 Hz
measured by using impulse oscillometry (R5-R20)
values. Patients with a high inferior-superior ΔHU gradient demonstrated evidence
of reduced specific ventilation in the lower lobes of the lungs and high levels
of PRMfSAD. A computational small-airway tree model
confirmed that constriction of gravitationally dependent, lower-zone,
small-airway branches would promote the largest increases in R5-R20 values.
Ventilation gradients correlated with asthma control and quality of life but
not with exacerbation frequency.

Conclusions

Lower lobe–predominant small-airways disease is a major driver of clinically
measured VH in adults with asthma.

A.N. Gorban, **Singularities
of transient processes in dynamics and beyond: Comment on "Long transients
in ecology: Theory and applications" by Andrew Morozov et al.** *Phys** Life Rev.* 2019 Dec 10, https://doi.org/10.1016/j.plrev.2019.12.002

From an absolutely rigorous (pedantic) point of
view, all the states in life or social sciences (and, possibly, far beyond
their borders) are transient. All the steady states, running waves, beautiful
limit cycles or attractors are just *intermediate asymptotics* and
nothing is stationary or ergodic.

However, the idea of *separation of time scales*
allows the creation of autonomous dynamic models at some scales. In these ideal
models, we assume that processes that are much faster than the selected time
scale are completely relaxed and the values of fast variables (or average
values in fast dynamics) follow the dynamics at the selected scale. The
processes proceeding much more slowly than the selected time can be considered
stationary or presented as a slow drift of the parameters of our models. This
clear and transparent picture is true if the system is globally stable and far
from a critical transition. Such a system relaxes to the limit regime without
significant delays, and all long transients are due to a slow drift of
parameters. When the qualitative picture of dynamics is not so trivial, then
various dynamic causes of slow relaxations and critical delay effects,
well-known in physics and beyond, may appear (we refer here to a recent review
of *critical transitions* [1]). Critical transitions can *mix time scales* and violate the standard separation of time logics. Even
violation of global stability can cause long delay near unstable attractors.

If we observe a long transition period for a real system, a conundrum arises:

*-is this delay caused by the drift of
“external” conditions (parameters),
-or does it have an internal dynamic cause,
-or does it appear just due to the inaccuracy of the model, because we
incorrectly determine the limit behaviour and transients?;…*

I.Y. Tyukin, A.N. Gorban, A. McEwan, S. Meshkinfamfard, **Bringing
the Blessing of Dimensionality to the Edge**. In *2019 1st International Conference on
Industrial Artificial Intelligence (IAI) *(pp. 1-5). IEEE (2019, July).

In this work we present a novel approach and
algorithms for equipping Artificial Intelligence systems with capabilities to
become better over time. A distinctive feature of the approach is that, in the
supervised setting, the approaches' computational complexity is sub-linear in
the number of training samples. This makes it particularly attractive in
applications in which the computational power and memory are limited. The
approach is based on the concentration of measure effects and stochastic
separation theorems. The algorithms are illustrated with examples.

H Chen, L Albergante,
JY Hsu, CA Lareau, GL Bosco, J Guan, S Zhou, AN
Gorban, DE Bauer, MJ Aryee, DM Langenau,
A Zinovyev, JD Buenrostro, G-C Yuan, L Pinello, **Single-cell trajectories reconstruction,
exploration and mapping of omics data with STREAM**.
Nature communications. 2019 Apr 23;10(1):1903. https://doi.org/10.1038/s41467-019-09670-4

Single-cell transcriptomic assays have enabled the de novo reconstruction of lineage differentiation trajectories, along with the characterization of cellular heterogeneity and state transitions. Several methods have been developed for reconstructing developmental trajectories from single-cell transcriptomic data, but efforts on analyzing single-cell epigenomic data and on trajectory visualization remain limited. Here we present STREAM, an interactive pipeline capable of disentangling and visualizing complex branching trajectories from both single-cell transcriptomic and epigenomic data. We have tested STREAM on several synthetic and real datasets generated with different single-cell technologies. We further demonstrate its utility for understanding myoblast differentiation and disentangling known heterogeneity in hematopoiesis for different organisms. STREAM is an open-source software package.

A.N. Gorban, A. Harel-Bellan, N. Morozova, A. Zinovyev, **Basic,
simple and extendable kinetic model of protein synthesis**, Mathematical
Biosciences and Engineering, 16, 6, 6602, 6622, 2019. https://doi.org/10.3934/mbe.2019329

Protein synthesis is one of the most fundamental biological processes. Despite existence of multiple mathematical models of translation, surprisingly, there is no basic and simple chemical kinetic model of this process, derived directly from the detailed kinetic scheme. One of the reasons for this is that the translation process is characterized by indefinite number of states, because of the structure of the polysome. We bypass this difficulty by applying lumping of multiple states of translated mRNA into few dynamical variables and by introducing a variable describing the pool of translating ribosomes. The simplest model can be solved analytically. The simplest model can be extended, if necessary, to take into account various phenomena such as the limited amount of ribosomal units or regulation of translation by microRNA. The introduced model is more suitable to describe the protein synthesis in eukaryotes but it can be extended to prokaryotes. The model can be used as a building block for more complex models of cellular processes. We demonstrate the utility of the model in two examples. First, we determine the critical parameters of the synthesis of a single protein for the case when the ribosomal units are abundant. Second, we demonstrate intrinsic bi-stability in the dynamics of the ribosomal protein turnover and predict that a minimal number of ribosomes should pre-exists in a living cell to sustain its protein synthesis machinery, even in the absence of proliferation.

I.Y.Tyukin,
A.Gorban, B. Grechuk, **Kernel
Stochastic Separation Theorems and Separability
Characterizations of Kernel Classifiers**,
proceeding of IJCNN 2019 - International Joint Conference on Neural Networks,
Budapest Hungary, 14-19 July 2019, paper N20219.

In this work we provide generalizations and extensions of stochastic separation theorems to kernel classifiers. A general separability result for two random sets is also established. We show that despite feature maps corresponding to a given kernel function may be infinite-dimensional, kernel separability characterizations can be expressed in terms of finite-dimensional volume integrals. These integrals allow to determine and quantify separability properties of an arbitrary kernel function. The theory is illustrated with numerical examples.

E.M.Mirkes,
J. Allohibi, A.N. Gorban, **Do
Fractional Norms and Quasinorms Help to Overcome the
Curse of Dimensionality?**,
proceeding of IJCNN 2019 - International Joint Conference on Neural Networks,
Budapest Hungary, 14-19 July 2019, paper N-19331.

The curse of dimensionality causes well-known and widely discussed problems for machine learning methods. There is a hypothesis that usage of Manhattan distance and even fractional quasinorms lp (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis for 37 binary classification problems on 25 databases.We confirm that fractional quasinorms have greater relative contrast or coefficient of variation than Euclidean norm l2, but we demonstrate also that the distance concentration shows qualitatively the same behaviour for all tested norms and quasinorms and the difference between them decays while dimension tends to infinity. Estimation of classification quality for kNN based on different norms and quasinorms shows that the greater relative contrast does not mean the better classifier performance and the worst performance for different databases was shown by the different norms (quasinorms). A systematic comparison shows that the difference in performance of kNN based on lp for p=2, 1, and 0.5 is statistically insignificant.

Gorban A.N., Makarov V.A., Tyukin I.Y. **Symphony
of high-dimensional brain: Reply to comments on** "The
unreasonable effectiveness of small neural ensembles in high-dimensional
brain". Phys Life Rev, 2019; https://doi.org/10.1016/j.plrev.2019.06.003. Preprint
version https://arxiv.org/pdf/1906.12222.pdf.

This paper is the final part
of the scientific discussion organised by the Journal
"Physics of Life Rviews" about the
simplicity revolution in neuroscience and AI. This discussion was initiated by
the review paper "The unreasonable effectiveness of small neural ensembles
in high-dimensional brain". Phys Life Rev 2019, https://doi.org/10.1016/j.plrev.2018.09.005 arXiv:1809.07656. The topics of the discussion varied from the
necessity to take into account the difference between the theoretical random
distributions and "extremely non-random" real distributions and
revise the common machine learning theory, to different forms of the curse of
dimensionality and high-dimensional pitfalls **in neuroscience.** V. Kůrková, A. Tozzi and J.F. Peters, R. Quian Quiroga, P. Varona, R. Barrio, G. Kreiman, L.
Fortuna, C. van Leeuwen, R. Quian Quiroga, and V. Kreinovich,
A.N. Gorban, V.A. Makarov, and I.Y. Tyukin participated in the discussion. In
this paper we analyse the symphony of opinions and
the possible outcomes of the simplicity revolution for machine learning and
neuroscience.

Tyukin, I. Y., Iudin,
D., Iudin, F., Tyukina, T., Kazantsev, V., Mukhina, I., & Gorban, A. N. (2019). **Simple
model of complex dynamics of activity patterns in developing networks of
neuronal cultures**. *PloS**
one*, *14*(6), e0218304 https://doi.org/10.1371/journal.pone.0218304.

Living neuronal networks in dissociated neuronal cultures are widely known for their ability to generate highly robust spatiotemporal activity patterns in various experimental conditions. Such patterns are often treated as neuronal avalanches that satisfy the power scaling law and thereby exemplify self-organized criticality in living systems. A crucial question is how these patterns can be explained and modeled in a way that is biologically meaningful, mathematically tractable and yet broad enough to account for neuronal heterogeneity and complexity. Here we derive and analyse a simple network model that may constitute a response to this question. Our derivations are based on few basic phenomenological observations concerning the input-output behavior of an isolated neuron. A distinctive feature of the model is that at the simplest level of description it comprises of only two variables, the network activity variable and an exogenous variable corresponding to energy needed to sustain the activity, and few parameters such as network connectivity and efficacy of signal transmission. The efficacy of signal transmission is modulated by the phenomenological energy variable. Strikingly, this simple model is already capable of explaining emergence of network spikes and bursts in developing neuronal cultures. The model behavior and predictions are consistent with published experimental evidence on cultured neurons. At the larger, cellular automata scale, introduction of the energy-dependent regulatory mechanism results in the overall model behavior that can be characterized as balancing on the edge of the network percolation transition. Network activity in this state shows population bursts satisfying the scaling avalanche conditions. This network state is self-sustainable and represents energetic balance between global network-wide processes and spontaneous activity of individual elements.

AN Gorban, VA Makarov, IY Tyukin, **The
unreasonable effectiveness of small neural ensembles in high-dimensional brain**, Physics of
Life Reviews, 2019, https://doi.org/10.1016/j.plrev.2018.09.005

Complexity
is an indisputable, well-known, and broadly accepted feature of the brain.
Despite the apparently obvious and widely-spread consensus on the brain
complexity, sprouts of the *single neuron revolution* emerged in
neuroscience in the 1970s. They brought many unexpected discoveries, including grandmother
or concept cells and sparse coding of information in the brain.

In machine learning for a long time, the famous curse of dimensionality seemed
to be an unsolvable problem. Nevertheless, the idea of the *blessing of
dimensionality* becomes gradually more and more popular. Ensembles of
non-interacting or weakly interacting simple units prove to be an effective
tool for solving essentially multidimensional and apparently incomprehensible
problems. This approach is especially useful for one-shot (non-iterative)
correction of errors in large legacy artificial intelligence systems and when
the complete re-training is impossible or too expensive.

These simplicity revolutions in the era of complexity have deep fundamental
reasons grounded in geometry of multidimensional data spaces. To explore and
understand these reasons we revisit the background ideas of statistical
physics. In the course of the 20th century they were developed into the
concentration of measure theory. The Gibbs equivalence of ensembles with
further generalizations shows that the data in high-dimensional spaces are
concentrated near shells of smaller dimension. New stochastic separation
theorems reveal the fine structure of the data clouds.

We review and analyse biological, physical, and
mathematical problems at the core of the fundamental question: how can
high-dimensional brain organise reliable and fast
learning in high-dimensional world of data by simple tools? To meet this
challenge, we outline and setup a framework based on statistical physics of
data.

Two critical applications are reviewed to exemplify the approach: one-shot
correction of errors in intellectual systems and emergence of static and
associative memories in ensembles of single neurons. Error correctors should be
simple; not damage the existing skills of the system; allow fast non-iterative
learning and correction of new mistakes without destroying the previous fixes.
All these demands can be satisfied by new tools based on the concentration of
measure phenomena and stochastic separation theory.

We show how a simple enough functional neuronal model is capable of explaining:
i) the extreme selectivity of single neurons to the
information content of high-dimensional data, ii) simultaneous separation of
several uncorrelated informational items from a large set of stimuli, and iii)
dynamic learning of new items by associating them with already “known” ones.
These results constitute a basis for organisation of
complex memories in ensembles of single neurons.

I.Y. Tyukin, A.N. Gorban, S. Green, D. Prokhorov,

**Fast Construction of Correcting Ensembles
for Legacy Artificial Intelligence Systems: Algorithms and a Case Study.
Information Sciences,** Volume 485, June 2019, Pages 230-247

This paper presents a new approach for constructing
simple and computationally efficient improvements of generic Artificial
Intelligence (AI) systems, including Multilayer and Deep Learning neural
networks. The improvements are small network ensembles added to the existing AI
architectures. Theoretical foundations of the approach are based on stochastic
separation theorems and the ideas of the concentration of measure. We show
that, subject to mild technical assumptions on statistical properties of internal
signals in the original AI system, the approach enables fast removal of the
AI’s errors with probability close to one on the datasets which may be
exponentially large in dimension. The approach is illustrated with numerical
examples and a case study of digits recognition in American Sign Language.

A.N. Gorban, R. Burton, I. Romanenko, I.Y. Tyukin,

**One-trial correction of legacy AI systems
and stochastic separation theorems**, Information Sciences
484 (2019) 237–254

We consider the problem of efficient “on the fly” tuning of existing, or
legacy, Artificial Intelligence (AI) systems. The legacy AI systems are allowed
to be of arbitrary class, albeit the data they are using for computing interim
or final decision responses should posses an
underlying structure of a high-dimensional topological real vector space. The
tuning method that we propose enables dealing with errors without the need to
re-train the sys- tem. Instead of re-training a simple cascade of perceptron
nodes is added to the legacy system. The added cascade modulates the AI legacy
system’s decisions. If applied repeatedly, the process results in a network of
modulating rules “dressing up” and improving performance of existing AI systems.
Mathematical rationale behind the method is based on the fundamental property
of measure concentration in high dimensional spaces. The method is illustrated
with an example of fine-tuning a deep convolutional network that has been
pre-trained to detect pedestrians in images.

*Inventors:* Ilya Romanenko, Ivan Tyukin, Alexander Gorban, Konstantin Sofeikov;

*Assignee:* Apical Ltd; Priority date:
2015-12-23. **Method of image processing**, *United States Patent*, Patent
No.: US 10,062,013 B2; Date of Patent: Aug. 28, 2018, published

According to an aspect of the present disclosure, there is provided a method of image processing. The method comprises receiving image data comprising a set of feature vectors of a first dimensionality, the feature vectors corresponding to a class of objects. A variable projection is applied to each feature vector in the set of feature vectors to generate a set of projected vectors of a second dimensionality. The method then comprises processing the set of projected vectors to generate a model for the class of objects. A projection is applied to the model to generate an object classification model, of the first dimensionality, for the class of objects.

S. Meshkinfamfard, A.
Gorban, I. Tyukin

**Tackling Rare False-Positives in Face
Recognition: A Case Study**. In 2018 IEEE 20th International Conference on High Performance
Computing and Communications; IEEE 16th International Conference on Smart City;
IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS) 2018 Jun 28 (pp. 1592-1598). IEEE.

In this study, we take on one of the most common challenges in facial recognition, i.e. reducing the False Positives in the recognition phase, through studying performance of a standard Deep Learning Convolutional network in a real-life, real-time, and large-scale identity surveillance application. This application involved designing a queue management system that uses facial recognition, for an airport in the UK. Our approach was to capture the faces of passengers as they enter through Boarding Pass Gates (BPG) and as they exit Security Gates (SG). Thereafter, we compare the faces captured, within a fifteen minute window, from BPG against the ones from SG. When there is a match, we are able to calculate the time that someone has spent inside the security area, using the capture time of matched face. We call this the security queue time. Like any other facial recognition application, we have to deal with reducing the number of false positives, i.e. incorrectly matched faces. In this application false positives are statistically rare events. That is, the same or similar pair of images is unlikely to occur in a foreseeable time. To deal with this problem, we utilized several approaches including applying a second layer of detection using the Dlib library to improve the quality of the detected faces. Specifically, by taking advantage of Dlibs Facial Landmarks, we created a scoring system similar to Dlibs, to choose the best frontal pose from amongst all faces attributed to a single person. Our large-scale trials show that this approach does measurably reduce the rate of false positives in such systems.

A.N. Gorban, A. Golubkov, B.
Grechuk, E.M. Mirkes, I.Y. Tyukin,

**Correction of AI systems by linear
discriminants: Probabilistic foundations**, *Information Sciences *466 (2018), 303-322.

Artificial Intelligence (AI) systems sometimes make errors and will make errors in the future, from time to time. These errors are usually unexpected, and can lead to dramatic consequences. Intensive development of AI and its practical applications makes the problem of errors more important. Total re-engineering of the systems can create new errors and is not always possible due to the resources involved. The important challenge is to develop fast methods to correct errors without damaging existing skills. We formulated the technical requirements to the ‘ideal’ correctors. Such correctors include binary classifiers, which separate the situations with high risk of errors from the situations where the AI systems work properly. Surprisingly, for essentially high-dimensional data such methods are possible: simple linear Fisher discriminant can separate the situations with errors from correctly solved tasks even for exponentially large samples. The paper presents the probabilistic basis for fast non-destructive correction of AI systems. A series of new stochastic separation theorems is proven. These theorems provide new instruments for fast non- iterative correction of errors of legacy AI systems. The new approaches become efficient in high-dimensions, for correction of high-dimensional systems in high-dimensional world (i.e. for processing of essentially high-dimensional data by large systems). We prove that this separability property holds for a wide class of distributions including log-concave distributions and distributions with a special ‘SMeared Absolute Continuity’ (SmAC) property defined through relations between the volume and probability of sets of vanishing volume. These classes are much wider than the Gaussian distributions. The requirement of independence and identical distribution of data is significantly relaxed. The results are supported by computational analysis of empirical data sets.

I. Tyukin, K. Sofeikov, J.
Levesley, A.N. Gorban, P. Allison, N.J. Cooper,

**Exploring Automated Pottery Identification
[Arch-I-Scan]**, Internet Archaeology 50 (2018). https://doi.org/10.11141/ia.50.11.

A hand-held smart device technology (Arch-I-Scan) is currently being developed and tested for scanning and classifying archaeological artefacts. The technology is based on a new platform developed by ARM, jointly with University of Leicester within Innovate UK Knowledge Transfer Partnership project (code KTP009890), and takes advantage of new algorithms for one-trial learning based on measure concentration phenomenon in high dimensions. This article discusses the development of a 'proof of concept' for automating the classification of Roman ceramic vessel types using whole vessels held in the collections of the Jewry Wall Museum, Leicester. The 'proof of concept' illustrates the viability and technical possibility of classifying and discriminating between objects of different types on-the-fly from a limited number of images. This technology is based on recent results (Gorban et al. 2016; Gorban and Tyukin 2017) revealing peculiar geometric properties of finite but large samples of data in high dimension. The ambition is to create a dedicated software that turns commonly available devices such as smart phones or tablets into scanners capable of classifying even small vessel sherds to the correct form and fabric.

I.Y. Tyukin, A.N. Gorban, K.I. Sofeykov,
I. Romanenko,

**Knowledge transfer between artificial
intelligence systems**, *Frontiers in Neurorobotics *12 (2018), https://doi.org/10.3389/fnbot.2018.00049

We consider the fundamental question: how a legacy “student” Artificial Intelligent (AI) system could learn from a legacy “teacher” AI system or a human expert without re-training and, most importantly, without requiring significant computational resources. Here “learning” is broadly understood as an ability of one system to mimic responses of the other to an incoming stimulation and vice-versa. We call such learning an Artificial Intelligence knowledge transfer. We show that if internal variables of the “student” Artificial Intelligent system have the structure of an n-dimensional topological vector space and n is sufficiently high then, with probability close to one, the required knowledge transfer can be implemented by simple cascades of linear functionals. In particular, for n sufficiently large, with probability close to one, the “student” system can successfully and non-iteratively learn k ≪ n new examples from the “teacher” (or correct the same number of mistakes) at the cost of two additional inner products. The concept is illustrated with an example of knowledge transfer from one pre-trained convolutional neural network to another.

I.Y. Tyukin, J.M. Al-Ameri, A.N. Gorban, J. Levesley, V.A. Terekhov,

**Fast Numerical Evaluation of Periodic
Solutions for a Class of Nonlinear Systems and Its Applications for Parameter
Estimation Problems.** In: Eremeev
A., Khachay M., Kochetov
Y., Pardalos P. (eds) Optimization Problems and Their Applications.
OPTA 2018. Communications in Computer and Information Science, vol 871. Springer, Cham, 2018, pp. 137-151.

Fast numerical evaluation of forward models is central for a broad range of inverse problems. Here we propose a method for deriving computationally efficient representations of periodic solutions of parameterized systems of nonlinear ordinary differential equations. These representations depend on parameters of the system explicitly, as quadratures of parameterized computable functions. The method applies to systems featuring both linear and nonlinear parametrization, and time-varying right-hand side. In addition, it opens possibilities to invoke scalable parallel computations and suitable function approximation schemes for numerical evaluation of solutions for various parameter values. Application of the method to the problem of parameter estimation of nonlinear ordinary differential equations is illustrated with a numerical example for the Morris–Lecar system.

A.N. Gorban, N. Çabukoǧlu,** Mobility cost and degenerated diffusion in kinesis models**, *Ecological Complexity* 36
(2018), 16-21.

A new critical effect is predicted in population dispersal. It is based on the
fact that a trade-off between the advantages of mobility and the cost of mobility
breaks with a significant deterioration in living conditions. The recently
developed model of purposeful kinesis (Gorban & Çabukoǧlu,
Ecological Complexity 33, 2018) is based on the “let well enough alone” idea:
mobility decreases for high reproduction coefficient and, therefore, animals
stay longer in good conditions and leave quicker bad conditions. Mobility has a
cost, which should be measured in the changes of the reproduction coefficient.
Introduction of the cost of mobility into the reproduction coefficient leads to
an equation for mobility. It can be solved in a closed form using Lambert
W-function. Surprisingly, the “let well enough alone” models with the simple
linear cost of mobility have an intrinsic phase transition: when conditions
worsen then the mobility increases up to some critical value of the
reproduction coefficient. For worse conditions, there is no solution for
mobility. We interpret this critical effect as the complete loss of mobility
that is degeneration of diffusion. Qualitatively, this means that mobility
increases with worsening of conditions up to some limit, and after that,
mobility is nullified.

A.N. Gorban, E.M. Mirkes, A. Zinovyev, **Data
analysis with arbitrary error measures approximated by piece-wise quadratic
PQSQ functions**, *Proceedings
of IJCNN 2018*, paper #18525.

Defining an error function (a measure of deviation of a model prediction from the data) is a critical step in any optimization-based data analysis method, including regression, clustering and dimension reduction. Usual quadratic error function in case of real-life high-dimensional and noisy data suffers from non- robustness to presence of outliers. Therefore, using non-quadratic error functions in data analysis and machine learning (such as L1 norm-based) is an active field of modern research but the majority of methods suggested are either slow or imprecise (use arbitrary heuristics). We suggest a flexible and highly performant approach to generalize most of existing data analysis methods to an arbitrary error function of subquadratic growth. For this purpose, we exploit PQSQ functions (piece-wise quadratic of subquadratic growth), which can be minimized by a simple and fast splitting-based iterative algorithm. The theoretical basis of the PQSQ approach is an application of min-plus (idempotent) algebra to data approximation. We introduce the general idea of the approach and illustrate it on four standard tools of machine learning: simple regression, regularized regression, $k$-mean clustering and principal component analysis. In all cases, PQSQ-based methods achieve better robustness with respect to the presence of strong noise in the data compared to the standard methods.

I.Y. Tyukin, A.N. Gorban, D. Prokhorov, S. Green, **Efficiency
of Shallow Cascades for Improving Deep Learning AI Systems**, *Proceedings of IJCNN 2018*, paper #18433.

This paper presents a technology for simple and non-iterative improvements of Multilayer and Deep Learning neural networks and Artificial Intelligence (AI) systems. The improvements are, in essence, shallow networks constructed on top of the existing Deep Learning architecture. Theoretical foundation of the technology is based on Stochastic Separation Theorems and the ideas of measure concentration. We show that, subject to mild technical assumptions on statistical properties of internal signals in Deep Learning AI, with probability close to one the technology enables instantaneous ''learning away'' of spurious and systematic errors. The method is illustrated with numerical examples.

J. Settipani, K. Karim, A. Chauvin, S.M. Ibnou-Ali, F. Paille-Barrere, E. Mirkes, A.N. Gorban, L. Larcombe, M.J. Whitcombe, T. Cowen, S.A. Piletsky.

**Theoretical
aspects of peptide imprinting: screening of MIP (virtual) binding sites for
their interactions with amino acids, di- and tripeptides.**** ***Journal
of the Chinese Advanced Materials Society, *2018. https://doi.org/10.1080/22243682.2018.1467279

Molecular modelling and computational approaches were used to design (virtual) molecularly imprinted binding sited for 170 amino acids, dipeptides and tripeptides. Analysis of the binding energy of ligands to their corresponding virtual binding sites revealed a direct correlation between size of the ligand and its binding affinity. Only tripeptides were capable of forming binding sites in molecularly imprinted polymers (MIPs) that are capable, in theory, of binding the corresponding targets at micromolar concentrations. No appreciable specificity was demonstrated in binding of virtual binding sites and corresponding templates. It is possible to conclude that although tripeptide sequences are sufficiently long to form MIPs with relatively high affinity, the sequence of peptide epitopes should be substantially longer that three amino acid residues to ensure specificity of imprinted sites. This consideration will be useful for the design of highly efficient MIPs for proteins.

A.N. Gorban.

**Model
reduction in chemical dynamics: slow invariant manifolds, singular
perturbations, thermodynamic estimates, and analysis of reaction graph.** *Current Opinion in Chemical Engineering*,
Volume 21, September 2018, 48-59.

The paper has two goals:

(1) It presents basic ideas, notions, and methods for reduction of reaction kinetics models: quasi-steady-state, quasi-equilibrium, slow invariant manifolds, and limiting steps.

(2) It describes briefly the current state of the art and some latest achievements in the broad area of model reduction in chemical and biochemical kinetics, including new results in methods of invariant manifolds, computation singular perturbation, bottleneck methods, asymptotology, tropical equilibration, and reaction mechanism skeletonization.

I. Tyukin, A.N. Gorban, C. Calvo, J. Makarova, V.A. Makarov.

**High-Dimensional
Brain: A Tool for Encoding and Rapid Learning of Memories by Single Neurons,** *Bull Math Biol*,
2018, https://doi.org/10.1007/s11538-018-0415-5

Codifying memories is one of the fundamental problems of modern Neuroscience. The functional mechanisms behind this phenomenon remain largely unknown. Experimental evidence suggests that some of the memory functions are performed by stratified brain structures such as the hippocampus. In this particular case, single neurons in the CA1 region receive a highly multidimensional input from the CA3 area, which is a hub for information processing. We thus assess the implication of the abundance of neuronal signalling routes converging onto single cells on the information processing. We show that single neurons can selectively detect and learn arbitrary information items, given that they operate in high dimensions. The argument is based on stochastic separation theorems and the concentration of measure phenomena. We demonstrate that a simple enough functional neuronal model is capable of explaining: (i) the extreme selectivity of single neurons to the information content, (ii) simultaneous separation of several uncorrelated stimuli or informational items from a large set, and (iii) dynamic learning of new items by associating them with already “known” ones. These results constitute a basis for organization of complex memories in ensembles of single neurons. Moreover, they show that no a priori assumptions on the structural organization of neuronal ensembles are necessary for explaining basic concepts of static and dynamic memories.

A.N. Gorban.

**Hilbert's
Sixth Problem: the endless road to rigour.** *Philosophical Transactions of The Royal Society A* 376(2118), 20170238, 2018.

Introduction to
the special issue of Phil. Trans. R. Soc. A 376, 2018, **`Hilbert's Sixth Problem'**. The essence of
the Sixth Problem is discussed and the content of this issue is introduced. In
1900, David Hilbert presented 23 problems for the advancement of mathematical
science. Hilbert's Sixth Problem proposed the expansion of the axiomatic method
outside of mathematics, in physics and beyond. Its title was shocking: "Mathematical
Treatment of the Axioms of Physics." Axioms of physics did not exist and
were not expected. During further explanation, Hilbert specified this problem
with special focus on probability and "the limiting processes,
... which lead from the atomistic view to the laws of motion of
continua". The programmatic call was formulated "to treat, by means
of axioms, those physical sciences in which already today mathematics plays an
important part." This issue presents a modern slice of the work on the
Sixth Problem, from quantum probability to fluid dynamics and machine learning,
and from review of solid mathematical and physical results to opinion pieces
with new ambitious ideas. Some expectations were broken: The continuum limit of
atomistic kinetics may differ from the classical fluid dynamics. The
"curse of dimensionality" in machine learning turns into the
"blessing of dimensionality" that is closely related to statistical
physics. Quantum probability facilitates the modelling of geological
uncertainty and hydrocarbon reservoirs. And many other findings are presented.

A.N. Gorban, I.Y. Tyukin.

**Blessing
of dimensionality: mathematical foundations of the statistical physics of data.** *Philosophical Transactions of The Royal Society A* 376(2118), 20170237, 2018.

The concentrations of measure phenomena were discovered as the mathematical background to statistical mechanics at the end of the nineteenth/beginning of the twentieth century and have been explored in mathematics ever since. At the beginning of the twenty-first century, it became clear that the proper utilization of these phenomena in machine learning might transform the curse of dimensionality into the blessing of dimensionality. This paper summarizes recently discovered phenomena of measure concentration which drastically simplify some machine learning problems in high dimension, and allow us to correct legacy artificial intelligence systems. The classical concentration of measure theorems state that i.i.d. random points are concentrated in a thin layer near a surface (a sphere or equators of a sphere, an average or median-level set of energy or another Lipschitz function, etc.). The new stochastic separation theorems describe the thin structure of these thin layers: the random points are not only concentrated in a thin layer but are all linearly separable from the rest of the set, even for exponentially large random sets. The linear functionals for separation of points can be selected in the form of the linear Fisher’s discriminant. All artificial intelligence systems make errors. Nondestructive correction requires separation of the situations (samples) with errors from the samples corresponding to correct behaviour by a simple and robust classifier. The stochastic separation theorems provide us with such classifiers and determine a noniterative (one-shot) procedure for their construction.

A.N. Gorban, N. Çabukoǧlu,

**Basic
model of purposeful kinesis**, *Ecological
Complexity*, 33, 2018, 75-83.

The notions of taxis and kinesis are introduced and
used to describe two types of behaviour of an organism in non-uniform
conditions: (i) Taxis means the guided movement to
more favourable conditions; (ii) Kinesis is the non-directional change in space
motion in response to the change of conditions. Migration and dispersal of
animals has evolved under control of natural selection. In a simple
formalisation, the strategy of dispersal should increase Darwinian fitness. We
introduce new models of purposeful kinesis with diffusion coefficient dependent
on fitness. The local and instant evaluation of Darwinian fitness is used, the
reproduction coefficient. New models include one additional parameter,
intensity of kinesis, and may be considered as the *minimal models of purposeful kinesis*. The properties of models are explored by a series of
numerical experiments. It is demonstrated how kinesis could be beneficial for
assimilation of patches of food or of periodic fluctuations. Kinesis based on
local and instant estimations of fitness is not always beneficial: for species
with the Allee effect it can delay invasion and
spreading. It is proven that kinesis cannot modify stability of homogeneous
positive steady states.

I.Y. Tyukin, A.N. Gorban, C. Calvo, J. Makarova, V.A. Makarov,

**High-dimensional brain. A tool for
encoding and rapid learning of memories by single neurons, ****arXiv:1710.11227**** [q-bio.NC]**

Codifying memories is one of the fundamental problems of modern Neuroscience. The functional mechanisms behind remain largely unknown. An experimental evidence suggests that some of the memory functions are performed by stratified brain structures as, e.g., the hippocampus. In this particular case, single neurons in the CA1 region receive a highly multidimensional input from the CA3 area, which is a hub for information processing. We thus assess the implication of the abundance of neuronal signaling routs converging onto single cells on the information processing. We show that single neurons can selectively detect and learn arbitrary information items, given that they operate in high dimensions. The argument is based on Stochastic Separation Theorems and measure concentration phenomena. We demonstrate that a simple enough functional neuronal model is capable of explaining: i) the extreme selectivity of single neurons to the information content, ii) simultaneous separation of several uncorrelated stimuli, or informational items, from a large set, and iii) dynamic learning of new items by associating them with already "known" ones. These results constitute a basis for organization of complex memories in ensembles of single neurons. Moreover, they show that no a priori assumptions on the structural organization of neuronal ensembles are necessary for explaining basic concepts of static and dynamic memories.

E.M. Mirkes, A.N. Gorban, J. Levesley, P.A.S. Elkington, J.A. Whetton,

**Pseudo-outcrop
Visualization of Borehole Images and Core Scans****, ***Mathematical
Geosciences,* 2017, November 2017, Volume 49, Issue 8, pp 947–964, https://doi.org/10.1007/s11004-017-9701-2

A pseudo-outcrop visualization is demonstrated for borehole and fulldiameter rock core images to augment the ubiquitous unwrapped cylinder view and thereby assist nonspecialist interpreters. The pseudo-outcrop visualization is equivalent to a nonlinear projection of the image from borehole to earth frame of reference that creates a solid volume sliced longitudinally to reveal two or more faces in which the orientations of geological features indicate what is observed in the subsurface. A proxy for grain size is used to modulate the external dimensions of the plot to mimic profiles seen in real outcrops. The volume is created from a mixture of geological boundary elements and texture, the latter being the residue after the sum of boundary elements is subtracted from the original data. In the case of measurements from wireline microresistivity tools, whose circumferential coverage is substantially <100%, the missing circumferential data are first inpainted using multiscale directional transforms, which decompose the image into its elemental building structures, before reconstructing the full image. The pseudo-outcrop view enables direct observation of the angular relationships between features and aids visual comparison between borehole and core images, especially for the interested nonspecialist.

Gorban A.N.,
Tyukin I.Y.

**Stochastic Separation Theorems****, ***Neural
Networks*, 94, October 2017, 255-259.

The problem of
non-iterative one-shot and non-destructive correction of unavoidable mistakes
arises in all Artificial Intelligence applications in the real world. Its
solution requires robust separation of samples with errors from samples where
the system works properly. We demonstrate that in (moderately) high dimension
this separation could be achieved with probability close to one by linear
discriminants. Based on fundamental properties of measure concentration, we
show that for M<aexp(bn) random M-element sets in R^{n}
are linearly separable with probability p, p>1−ϑ, where 1>ϑ>0 is a
given small constant. Exact values of a,b>0
depend on the probability distribution that determines how the random M-element
sets are drawn, and on the constant ϑ. These stochastic separation theorems
provide a new instrument for the development, analysis, and assessment of
machine learning methods and algorithms in high dimension. Theoretical
statements are illustrated with numerical examples.

Fehrman E., Muhammad A.K., Mirkes E.M., Egan V., Gorban
A.N.

**The**** Five Factor Model of Personality and Evaluation of Drug Consumption
Risk.**** **In:
Palumbo F., Montanari A., Vichi
M. (eds) Data Science.
Studies in Classification, Data Analysis, and Knowledge Organization. Springer
(2017), pp 231-242.

The problem of evaluating an individual’s risk of drug consumption and misuse
is highly important and novel. An online survey methodology was employed to
collect data including personality traits (NEO-FFI-R), impulsivity (BIS-11),

sensation seeking (ImpSS), and demographic information. The data set contained information on the consumption of 18 central nervous system psychoactive drugs. Correlation analysis using a relative information gain model demonstrates the existence of a group of drugs (amphetamines, cannabis, cocaine, ecstasy, legal highs, LSD, and magic mushrooms) with strongly correlated consumption. An exhaustive search was performed to select the most effective subset of input features and data mining methods to classify users and non-users for each drug. A number of classification methods were employed (decision tree, random forest, k-nearest neighbours, linear discriminant analysis, Gaussian mixture, probability density function estimation, logistic regression, and naïve Bayes) and the most effective method selected for each drug. The quality of classification was surprisingly high. The best results with sensitivity and specificity being greater than 75% were achieved for cannabis, crack, ecstasy, legal highs, LSD, and volatile substance abuse. Sensitivity and specificity greater than 70% were achieved for amphetamines, amyl nitrite, benzodiazepines, chocolate, caffeine, heroin, ketamine, methadone, and nicotine.

A.N. Gorban,
I.V. Karlin

**Beyond Navier–Stokes
equations: capillarity of ideal gas****,** *Contemporary
Physics,* 58(1) (2017), 70-90, DOI:10.1080/00107514.2016.1256123. **arXiv**** e-print**

The system of Navier–Stokes–Fourier equations is one of the most celebrated systems of equations in modern science. It describes dynamics of fluids in the limit when gradients of density, velocity and temperature are sufficiently small, and loses its applicability when the flux becomes so non-equilibrium that the changes of velocity, density or temperature on the length compatible with the mean free path are non-negligible. The question is: how to model such fluxes? This problem is still open. (Despite the fact that the first ‘final equations of motion’ modified for analysis of thermal creep in rarefied gas were proposed by Maxwell in 1879.) There are, at least, three possible answers: (i) use molecular dynamics with individual particles, (ii) use kinetic equations, like Boltzmann’s equation or (iii) find a new system of equations for description of fluid dynamics with better accounting of non-equilibrium effects. These three approaches work at different scales. We explore the third possibility using the recent findings of capillarity of internal layers in ideal gases and of saturation effect in dissipation (there is a limiting attenuation rate for very short waves in ideal gas and it cannot increase infinitely). One candidate equation is discussed in more detail, the Korteweg system proposed in 1901. The main ideas and approaches are illustrated by a kinetic system for which the problem of reduction of kinetics to fluid dynamics is analytically solvable.

** **

**2016**

A.N. Gorban, I.Y. Tyukin, I. Romanenko,

**The**** Blessing of Dimensionality: Separation
Theorems in the Thermodynamic Limit**, *IFAC-PapersOnLine* 49-24 (2016), 064–069.

We consider and analyze properties of large sets of
randomly selected (i.i.d.) points in high dimensional
spaces. In particular, we consider the problem of whether a single data point
that is randomly chosen from a finite set of points can be separated from the
rest of the data set by a linear hyperplane. We formulate and prove stochastic
separation theorems, including: 1) with
probability close to one a random point may be separated from a finite random
set by a linear functional; 2) with probability close to one for every point in
a finite random set there is a linear functional separating this point from the
rest of the data. The total number of points in the random sets are allowed to
be exponentially large with respect to dimension. Various laws governing
distributions of points are considered, and explicit formulae for the
probability of separation are provided. These theorems reveal an interesting
implication for machine learning and data mining applications that deal with
large data sets (big data) and high-dimensional data (many attributes): simple
linear decision rules and learning machines are surprisingly efficient tools
for separating and filtering out arbitrarily assigned points in large
dimensions.

E. Moczko, E.M. Mirkes, C. Cáceres, A.N. Gorban, S. Piletsky,

**Fluorescence-based
assay as a new screening tool for toxic chemicals**, *Scientific Reports*
6, Article number: 33922 (2016)

Our study involves development of fluorescent cell-based diagnostic assay as a new approach in high-throughput screening method. This highly sensitive optical assay operates similarly to e-noses and e-tongues which combine semi-specific sensors and multivariate data analysis for monitoring biochemical processes. The optical assay consists of a mixture of environmental-sensitive fluorescent dyes and human skin cells that generate fluorescence spectra patterns distinctive for particular physico-chemical and physiological conditions. Using chemometric techniques the optical signal is processed providing qualitative information about analytical characteristics of the samples. This integrated approach has been successfully applied (with sensitivity of 93% and specificity of 97%) in assessing whether particular chemical agents are irritating or not for human skin. It has several advantages compared with traditional biochemical or biological assays and can impact the new way of high-throughput screening and understanding cell activity. It also can provide reliable and reproducible method for assessing a risk of exposing people to different harmful substances, identification active compounds in toxicity screening and safety assessment of drugs, cosmetic or their specific ingredients.

A.N. Gorban, E.M. Mirkes, A. Zinovyev,

**Piece-wise
quadratic approximations of arbitrary error functions for fast and robust
machine learning****,**
Neural Networks, Volume 84, December 2016, 28-38

Most of machine learning approaches have stemmed from the application of minimizing the mean squared distance principle, based on the computationally efficient quadratic optimization methods. However, when faced with high-dimensional and noisy data, the quadratic error functionals demonstrated many weaknesses including high sensitivity to contaminating factors and dimensionality curse. Therefore, a lot of recent applications in machine learning exploited properties of non-quadratic error functionals based on L_1 norm or even sub-linear potentials corresponding to quasinorms L_p (0<p<1). The back side of these approaches is increase in computational cost for optimization. Till so far, no approaches have been suggested to deal with arbitrary error functionals, in a flexible and computationally efficient framework. In this paper, we develop a theory and basic universal data approximation algorithms (k-means, principal components, principal manifolds and graphs, regularized and sparse regression), based on piece-wise quadratic error potentials of subquadratic growth (PQSQ potentials). We develop a new and universal framework to minimize arbitrary sub-quadratic error potentials using an algorithm with guaranteed fast convergence to the local or global error minimum. The theory of PQSQ potentials is based on the notion of the cone of minorant functions, and represents a natural approximation formalism based on the application of min-plus algebra. The approach can be applied in most of existing machine learning methods, including methods of data approximation and regularized and sparse regression, leading to the improvement in the computational cost/accuracy trade-off. We demonstrate that on synthetic and real-life datasets PQSQ-based machine learning methods achieve orders of magnitude faster computational performance than the corresponding state-of-the-art methods, having similar or better approximation accuracy.

E.M. Mirkes, T.J. Coats, J. Levesley, A.N. Gorban,

**Handling missing
data in large healthcare dataset: a case study of unknown trauma outcomes****, **

Computers in Biology and Medicine 75 (2016), 203-216.

Handling of missed data is one of the main tasks in data preprocessing especially in large public service datasets. We have analysed data from the Trauma Audit and Research Network (TARN) database, the largest trauma database in Europe. For the analysis we used 165,559 trauma cases. Among them, there are 19,289 cases (13.19%) with unknown outcome. We have demonstrated that these outcomes are not missed ‘completely at random’ and, hence, it is impossible just to exclude these cases from analysis despite the large amount of available data. We have developed a system of non-stationary Markov models for the handling of missed outcomes and validated these models on the data of 15,437 patients which arrived into TARN hospitals later than 24 hours but within 30 days from injury. We used these Markov models for the analysis of mortality. In particular, we corrected the observed fraction of death. Two naïve approaches give 7.20% (available case study) or 6.36% (if we assume that all unknown outcomes are ‘alive’). The corrected value is 6.78%. Following the seminal paper of Trunkey, the multimodality of mortality curves has become a much discussed idea. For the whole analysed TARN dataset the coefficient of mortality monotonically decreases in time but the stratified analysis of the mortality gives a different result: for lower severities the coefficient of mortality is a non-monotonic function of the time after injury and may have maxima at the second and third weeks. The approach developed here can be applied to various healthcare datasets which experience the problem of lost patients and missed outcomes.

A.N. Gorban, T.A. Tyukina, E.V. Smirnova, L.I. Pokidysheva

**Evolution
of adaptation mechanisms: Adaptation energy, stress, and oscillating death**,

Journal of Theoretical Biology, 405 (2016), 127-139, http://dx.doi.org/10.1016/j.jtbi.2015.12.017

• We formalize Selye׳s ideas about adaptation energy and dynamics of adaptation.

• A hierarchy of dynamic models of adaptation is developed.

• Adaptation energy is considered as an internal coordinate on the ‘dominant path’ in the model of adaptation.

• The optimal distribution of resources for neutralization of harmful factors is studied.

• The phenomena of ‘oscillating death’ and ‘oscillating remission’ are predicted.

In 1938, Selye proposed the notion of adaptation energy and published ‘Experimental evidence supporting the conception of adaptation energy.’ Adaptation of an animal to different factors appears as the spending of one resource. Adaptation energy is a hypothetical extensive quantity spent for adaptation. This term causes much debate when one takes it literally, as a physical quantity, i.e. a sort of energy. The controversial points of view impede the systematic use of the notion of adaptation energy despite experimental evidence. Nevertheless, the response to many harmful factors often has general non-specific form and we suggest that the mechanisms of physiological adaptation admit a very general and nonspecific description.

We aim to demonstrate that Selye׳s adaptation energy is the cornerstone of the top-down approach to modelling of non-specific adaptation processes. We analyze Selye׳s axioms of adaptation energy together with Goldstone׳s modifications and propose a series of models for interpretation of these axioms. Adaptation energy is considered as an internal coordinate on the ‘dominant path’ in the model of adaptation. The phenomena of ‘oscillating death’ and ‘oscillating remission’ are predicted on the base of the dynamical models of adaptation. Natural selection plays a key role in the evolution of mechanisms of physiological adaptation. We use the fitness optimization approach to study of the distribution of resources for neutralization of harmful factors, during adaptation to a multifactor environment, and analyze the optimal strategies for different systems of factors.

A.A. Akinduko, E.M. Mirkes, A.N. Gorban

**SOM: Stochastic initialization versus
principal components**

Information Sciences Volumes 364–365, 10 October 2016, Pages 213–221. http://dx.doi.org/10.1016/j.ins.2015.10.013

Selection of a good initial approximation is a well known problem for all iterative methods of data approximation, from k-means to Self-Organizing Maps (SOM) and manifold learning. The quality of the resulting data approximation depends on the initial approximation. Principal components are popular as an initial approximation for many methods of nonlinear dimensionality reduction because its convenience and exact reproducibility of the results. Nevertheless, the reports about the results of the principal component initialization are controversial.

In this work, we separate datasets into two classes: quasilinear and essentially nonlinear datasets. We demonstrate on learning of one-dimensional SOM (models of principal curves) that for the quasilinear datasets the principal component initialization of the self-organizing maps is systematically better than the random initialization, whereas for the essentially nonlinear datasets the random initialization may perform better. Performance is evaluated by the fraction of variance unexplained in numerical experiments.

A.N. Gorban, I.Yu. Tyukin, D.V. Prokhorov, K.I. Sofeikov

**Approximation with random bases: Pro et Contra**

Information Sciences Volumes 364–365, 10 October 2016, Pages 129–145. http://dx.doi.org/10.1016/j.ins.2015.09.021

In this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a Hilbert space of functions. We consider and analyze published procedures, both randomized and deterministic, for selecting elements from these families that have been shown to ensure the rate of convergence in L2 norm of order O(1/N), where N is the number of elements. We show that both randomized and deterministic procedures are successful if additional information about the families of functions to be approximated is provided. In the absence of such additional information one may observe exponential growth of the number of terms needed to approximate the function and/or extreme sensitivity of the outcome of the approximation to parameters. Implications of our analysis for applications of neural networks in modeling and control are illustrated with examples.

** **

A.N. Gorban, N. Jarman, E. Steur, H. Nijmeijer, C. van Leeuwen, I. Tyukin

**Directed
cycles and multi-stability of coherent dynamics in systems of coupled nonlinear
oscillators**, IFAC-PapersOnLine,
48, (18) (2015), 19–24,

We analyse the dynamics of networks of coupled nonlinear systems in terms of both topology of interconnections as well as the dynamics of individual nodes. Here we focus on two basic and extremal components of any network: chains and cycles. In particular, we investigate the effect of adding a directed feedback from the last element in a directed chain to the first. Our analysis shows that, depending on the network size and internal dynamics of isolated nodes, multiple coherent and orderly dynamic regimes co-exist in the state space of the system. In addition to the fully synchronous state an attracting rotating wave solution occurs. The basin of attraction of this solution apparently grows with the number of nodes in the loop. The effect is observed in networks exceeding a certain critical size. Emergence of the attracting rotating wave solution can be viewed as a “topological bifurcation” of network dynamics in which removal or addition of a single connection results in dramatic change of the overall coherent dynamics of the system.

A. N. Gorban,·A. Zinovyev

**Fast and
user-friendly non-linear principal manifold learning by method of elastic maps**,
in Proceedings DSAA 2015 -- IEEE International Conference on Data Science and
Advanced Analytics, Paris; 10/2015

Method of elastic maps allows fast learning of non-linear principal manifolds for large datasets. We present user-friendly implementation of the method in ViDaExpert software. Equipped with several dialogs for configuring data point representations (size, shape, color) and fast 3D viewer, ViDaExpert is a handy tool allowing to construct an interactive 3D-scene representing a table of data in multidimensional space and perform its quick and insightfull statistical analysis, from basic to advanced methods. We list several recent application examples of manifold learning by method of elastic maps in various fields of life sciences.

** **

A.N.Gorban,

**Forward-Invariant
Peeling in Chemical Dynamics: a Simple Case Study****,** Math. Model. Nat. Phenom. Vol. 10,
No. 5, 2015, pp. 126–134.

Forward-invariant peeling aims to produce forward-invariant subset from a given set in phase space. The structure of chemical kinetic equations allows us to describe the general operations of the forward-invariant peeling. For example, we study a simple reaction network with three components A1,A2,A3 and reactions $A1 \to A2 \to A3 \to A1$, $2A1 \leftrightarrows 3A2$ (without any stoichiometric conservation law). We assume that kinetics obey the classical mass action law and reaction rate constants are positive intervals $0 < min ki \leq ki \leq max ki < 1$. Kinetics of this system is described by a system of differential inclusions. We produce forward-invariant sets for these kinetic inclusions from the sets ${c|ci\geq 0,\sum ci \geq \epsilon}$ by the forward-invariant peeling (for sufficiently small $\epsilon > 0$). In particular, this construction proves persistence of this kinetic system (a positive solution cannot approach the origin even asymptotically, as $t\to \infty$).

A.N. Gorban, V.N. Kolokoltsov,

**Generalized Mass Action Law and
Thermodynamics of Nonlinear Markov Processes**, Math. Model. Nat.
Phenom. Vol. 10, No. 5, 2015, pp. 16–46

The nonlinear Markov processes are measure-valued dynamical systems which preserve positivity. They can be represented as the law of large numbers limits of general Markov models of interacting particles. In physics, the kinetic equations allow Lyapunov functionals (entropy, free energy, etc.). This may be considered as a sort of inheritance of the Lyapunov functionals from the microscopic master equations. We study nonlinear Markov processes that inherit thermodynamic properties from the microscopic linear Markov processes. We develop the thermodynamics of nonlinear Markov processes and analyze the asymptotic assumption, which are sufficient for this inheritance.

A.N. Gorban, G.S. Yablonsky,

**Three
Waves of Chemical Dynamics****,**
Math. Model. Nat. Phenom. Vol. 10, No. 5, 2015, pp. 1–5.

Three epochs in development of chemical dynamics are presented. We try to understand the modern research programs in the light of classical works. Three eras (or waves) of chemical dynamics can be revealed in the flux of research and publications. These waves may be associated with leaders: the first is the van’t Hoff wave, the second may be called the

Semenov–Hinshelwood wave and the third is definitely the Aris wave. The ‘waves’ may be distinguished based on the main focuses of the scientific leaders:

– Van’t Hoff was searching for the general law of chemical reaction related to specific chemical properties. The term “chemical dynamics” belongs to van’t Hoff.

– The Semenov-Hinshelwood focus was an explanation of critical phenomena observed in many chemical systems, in particular in flames. A concept “chain reactions” elaborated by these researchers influenced many sciences, especially nuclear physics and engineering.

– Aris’ activity was concentrated on the detailed systematization of mathematical ideas and approaches.

** **

A.N. Gorban, N. Jarman,
E. Steur, C. van Leeuwen, I.Yu.
Tyukin

**Leaders Do Not Look Back, or Do They?**** ***Math.
Model. Nat. Phenom.* Vol. 10, No. 3, 2015, pp. 212–231.

We study the effect of adding to a directed chain
of interconnected systems a directed feedback from the last element in the
chain to the first. The problem is closely related to the fundamental question
of how a change in network topology may influence the behavior
of coupled systems. We begin the analysis by investigating a simple linear
system. The matrix that specifies the system dynamics is the transpose of the
network Laplacian matrix, which codes the connectivity of the network. Our
analysis shows that for any nonzero complex eigenvalue λ of this matrix, the
following inequality holds: |Im λ|/|Re λ|≤cot(π/n). This bound is sharp, as it becomes an equality for
an eigenvalue of a simple directed cycle with uniform interaction weights. The
latter has the slowest decay of oscillations among all other network
configurations with the same number of states. The result is generalized to
directed rings and chains of identical nonlinear oscillators. For directed
rings, a lower bound σ_{c} for the connection
strengths that guarantees asymptotic synchronization is found to follow a
similar pattern: σ_{c} = 1/(1−cos(2π/n)).
Numerical analysis revealed that, depending on the network size n, multiple
dynamic regimes co-exist in the state space of the system. In addition to the
fully synchronous state a rotating wave solution occurs. The effect is observed
in networks exceeding a certain critical size. The emergence of a rotating wave
highlights the importance of long chains and loops in networks of oscillators:
the larger the size of chains and loops, the more sensitive the network
dynamics becomes to removal or addition of a single connection.

** **

** **

** **

A.N. Gorban, I.Yu. Tyukin, H. Nijmeijer

**Further Results on Lyapunov-Like
Conditions of Forward Invariance and Boundedness for a Class of Unstable
Systems,** in Proceedings of 53rd IEEE Conference on
Decision and Control December 15-17, 2014. Los Angeles, California, USA, IEEE,
2014, pp. 1557-1562

We provide several characterizations of
convergence to unstable equilibria in nonlinear systems. Our current
contribution is three-fold. First we present simple algebraic conditions for
establishing local convergence of non-trivial solutions of nonlinear systems to
unstable equilibria. The conditions are based on our earlier work [A.N. Gorban,
I. Tyukin, E. Steur, and H. Nijmeijer
**Lyapunov****-like conditions of forward invariance and
boundedness for a class of unstable systems**, *SIAM J. Control Optim*., Vol. 51, No. 3,
2013, pp. 2306-2334.] and can be viewed as an extension of the Lyapunov’s first method in that they apply to systems in
which the corresponding Jacobian has one zero eigenvalue. Second, we show that
for a relevant subclass of systems, persistency of excitation of a function of
time in the right-hand side of the equations governing dynamics of the system
ensure existence of an attractor basin such that solutions passing through this
basin in forward time converge to the origin exponentially. Finally we
demonstrate that conditions developed earlier [A.N. Gorban, I. Tyukin, E. Steur, and H. Nijmeijer, *SIAM J. Control Optim*.,
Vol. 51, No. 3, 2013, pp. 2306-2334.] may be remarkably tight.

** **

A.S. Manso, M.H. Chai, J.M.
Atack, L. Furi, M. De Ste Croix, R. Haigh, C. Trappetti,
A.D. Ogunniyi, L.K. Shewell,
M. Boitano, T.A. Clark, J. Korlach,
M. Blades, E. Mirkes, A.N. Gorban, J.C. Paton, M.P. Jennings, M.R. Oggioni

**A
random six-phase switch regulates pneumococcal virulence via global epigenetic
changes**, *Nature
Communications* 5 (2014), Article number: 5055. DOI: 10.1038/ncomms6055. **Supplementary Information**

Streptococcus pneumoniae (the pneumococcus) is the
world’s foremost bacterial pathogen in both morbidity and mortality. Switching
between phenotypic forms (or ‘phases’) that favour
asymptomatic carriage or invasive disease was first reported in 1933. Here, we
show that the underlying mechanism for such phase variation consists of genetic
rearrangements in a Type I restriction-modification system (SpnD39III). The
rearrangements generate six alternative specificities with distinct methylation
patterns, as defined by single-molecule, real-time (SMRT) methylomics.
The SpnD39III variants have distinct gene expression profiles. We demonstrate
distinct virulence in experimental infection and in vivo selection for
switching between SpnD39III variants. SpnD39III is ubiquitous in pneumococci,
indicating an essential role in its biology. Future studies must recognize the
potential for switching between these heretofore undetectable, differentiated
pneumococcal subpopulations in vitro and in vivo. Similar systems exist in
other bacterial genera, indicating the potential for broad exploitation of
epigenetic gene regulation.

E.M. Mirkes, I. Alexandrakis,
K. Slater, R. Tuli, A.N. Gorban

**Computational diagnosis and risk
evaluation for canine lymphoma***, Computers in Biology and Medicine*,
Volume 53, 1 October 2014, 279-290.

· Acute phase proteins, C-Reactive Protein and Haptoglobin, are used for the canine lymphoma blood test.

· This test can be used for diagnostics, screening, and for remission monitoring.

· We compare various decision trees, KNN (and advanced KNN) and algorithms for probability density evaluation.

· For the differential diagnosis the best solution gives the sensitivity 83.5% and specificity 77%.

The canine lymphoma blood test detects the levels of two biomarkers, the acute phase proteins (C-Reactive Protein and Haptoglobin). This test can be used for diagnostics, for screening, and for remission monitoring as well. We analyze clinical data, test various machine learning methods and select the best approach to these oblems. Three families of methods, decision trees, kNN (including advanced and adaptive kNN) and probability density evaluation with radial basis functions, are used for classification and risk estimation. Several pre-processing approaches were implemented and compared. The best of them are used to create the diagnostic system. For the differential diagnosis the best solution gives the sensitivity and specificity of 83.5% and 77%, respectively (using three input features, CRP, Haptoglobin and standard clinical symptom). For the screening task, the decision tree method provides the best result, with sensitivity and specificity of 81.4% and >99%, respectively (using the same input features). If the clinical symptoms (Lymphadenopathy) are considered as unknown then a decision tree with CRP and Hapt only provides sensitivity 69% and specificity 83.5%. The lymphoma risk evaluation problem is formulated and solved. The best models are selected as the system for computational lymphoma diagnosis and evaluation of the risk of lymphoma as well. These methods are implemented into a special web-accessed software and are applied to the problem of monitoring dogs with lymphoma after treatment. It detects recurrence of lymphoma up to two months prior to the appearance of clinical signs. The risk map visualization provides a friendly tool for exploratory data analysis.

A.N. Gorban

**Detailed balance in micro- and macrokinetics and
micro-distinguishability of macro-processes**, *Results
in Physics,* Volume 4, 2014, 142-147.

We develop a general framework for the discussion of detailed balance and analyse its microscopic background. We find that there should be two additions to the well-known T- or PT-invariance of the microscopic laws of motion:

1. Equilibrium should not spontaneously break the relevant T- or PT-symmetry.

2. The macroscopic processes should be microscopically distinguishable to guarantee persistence of detailed balance in the model reduction from micro- to macrokinetics.

We briefly discuss examples of the violation of these rules and the corresponding violation of detailed balance.

** **

K.I. Sofeikov, I. Yu. Tyukin, A.N.Gorban, E.M.Mirkes, D.V. Prokhorov, and I.V.Romanenko,

In Proceedings of 2014 International Joint Conference on Neural Networks (IJCNN) July 6-11, 2014, Beijing, China, IEEE 2014, pp. 3548-3555, URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6889842&isnumber=6889358

We consider the problem of construction of decision trees in cases when data are non-categorical and are inherently high-dimensional. Using conventional tree growing algorithms that either rely on univariate splits or employ direct search methods for determining multivariate splitting conditions is computationally prohibitive. On the other hand application of standard optimization methods for finding locally optimal splitting conditions is obstructed by abundance of local minima and discontinuities of classical goodness functions such as e.g. information gain or Gini impurity. In order to avoid this limitation a method to generate smoothed replacement for measuring impurity of splits is proposed. This enables to use vast number of efficient optimization techniques for finding locally optimal splits and, at the same time, decreases the number of local minima. The approach is illustrated with examples.

A.N. Gorban, D.J. Packwood

**Enhancement of the stability of lattice
Boltzmann methods by dissipation control****, **Physica A 414
(2014) 285–299, http://doi.org/10.1016/j.physa.2014.07.052

Artificial dissipation is a well known tool for the improvement of stability of numerical algorithms. However, the use of this technique affects the accuracy of the computation. We analyze various approaches proposed for enhancement of the Lattice Boltzmann Methods (LBM) stability. In addition to some previously known methods, the Multiple Relaxation Time (MRT) models, the entropic lattice Boltzmann method (ELBM), and filtering (including entropic median filtering), we develop and analyze new filtering techniques with independent filtering of different modes. All these methods affect dissipation in the system and may adversely affect the reproduction of the proper physics. To analyze the effect of dissipation on accuracy and to prepare practical recommendations, we test the enhanced LBM methods on the standard benchmark, the 2D lid driven cavity on a coarse grid (101××101 nodes). The accuracy was estimated by the position of the first Hopf bifurcation points in these systems. We find that two techniques, MRT and median filtering, succeed in yielding a reasonable value of the Reynolds number for the first bifurcation point. The newly created limiters, which filter the modes independently, also pick a reasonable value of the Reynolds number for the first bifurcation.

• The stability problem arises for lattice Boltzmann methods in modelling of highly non-equilibrium fluxes.

• Dissipation control is an efficient tool to improve stability but it affects accuracy.

• We analyse the stability–accuracy problem for lattice Boltzmann methods with additional dissipation.

• We compare various methods for dissipation control: Entropic filtering, Multirelaxation methods and Entropic collisions.

• For numerical test we use the lid driven cavity; the accuracy was estimated by the position of the first Hopf bifurcation.

A.N. Gorban,

**General H-theorem
and Entropies that Violate the Second Law**.

*H*-theorem states that the entropy production is nonnegative and,
therefore, the entropy of a closed system should monotonically change in time.
In information processing, the entropy production is positive for random
transformation of signals (the information processing lemma). Originally, the *H*-theorem and the information processing
lemma were proved for the classical Boltzmann-Gibbs-Shannon entropy and for the
correspondent divergence (the relative entropy). Many new entropies and
divergences have been proposed during last decades and for all of them the *H*-theorem is needed. This note proposes
a simple and general criterion to check whether the *H*-theorem is valid for a convex divergence *H* and demonstrates that some of the popular divergences obey no *H*-theorem. We consider systems with *n* states *A _{i}* that obey first order kinetics (master equation). A
convex function

E M Mirkes, I Alexandrakis, K Slater, R Tuli and A N Gorban,

**Computational diagnosis of canine lymphoma**, J. Phys.: Conf. Ser. 490 012135 (2014)

One out of four dogs will develop cancer in their lifetime and 20% of those will be lymphoma cases. PetScreen developed a lymphoma blood test using serum samples collected from several veterinary practices. The samples were fractionated and analysed by mass spectrometry. Two protein peaks, with the highest diagnostic power, were selected and further identified as acute phase proteins, C-Reactive Protein and Haptoglobin. Data mining methods were then applied to the collected data for the development of an online computer-assisted veterinary diagnostic tool. The generated software can be used as a diagnostic, monitoring and screening tool. Initially, the diagnosis of lymphoma was formulated as a classification problem and then later refined as a lymphoma risk estimation. Three methods, decision trees, kNN and probability density evaluation, were used for classification and risk estimation and several preprocessing approaches were implemented to create the diagnostic system. For the differential diagnosis the best solution gave a sensitivity and specificity of 83.5% and 77%, respectively (using three input features, CRP, Haptoglobin and standard clinical symptom). For the screening task, the decision tree method provided the best result, with sensitivity and specificity of 81.4% and >99%, respectively (using the same input features). Furthermore, the development and application of new techniques for the generation of risk maps allowed their user-friendly visualization.

A A Akinduko and A N Gorban,

**Multiscale principal component analysis**, J. Phys.: Conf. Ser. 490 012081 (2014)

Principal component analysis (PCA) is an important tool in exploring data. The conventional approach to PCA leads to a solution which favours the structures with large variances. This is sensitive to outliers and could obfuscate interesting underlying structures. One of the equivalent definitions of PCA is that it seeks the subspaces that maximize the sum of squared pairwise distances between data projections. This definition opens up more flexibility in the analysis of principal components which is useful in enhancing PCA. In this paper we introduce scales into PCA by maximizing only the sum of pairwise distances between projections for pairs of datapoints with distances within a chosen interval of values [l,u]. The resulting principal component decompositions in Multiscale PCA depend on point (l,u) on the plane and for each point we define projectors onto principal components. Cluster analysis of these projectors reveals the structures in the data at various scales. Each structure is described by the eigenvectors at the medoid point of the cluster which represent the structure. We also use the distortion of projections as a criterion for choosing an appropriate scale especially for data with outliers. This method was tested on both artificial distribution of data and real data. For data with multiscale structures, the method was able to reveal the different structures of the data and also to reduce the effect of outliers in the principal component analysis.

Y Shi, A N Gorban and T Y Yang,

**Is it possible to predict long-term success
with k-NN? Case study of four market indices (FTSE100, DAX, HANGSENG, NASDAQ)**, J. Phys.: Conf. Ser. 490 012082 (2014)

This case study tests the possibility of prediction for 'success' (or 'winner') components of four stock & shares market indices in a time period of three years from 02-Jul-2009 to 29-Jun-2012.We compare their performance ain two time frames: initial frame three months at the beginning (02/06/2009-30/09/2009) and the final three month frame (02/04/2012-29/06/2012).To label the components, average price ratio between two time frames in descending order is computed. The average price ratio is defined as the ratio between the mean prices of the beginning and final time period. The 'winner' components are referred to the top one third of total components in the same order as average price ratio it means the mean price of final time period is relatively higher than the beginning time period. The 'loser' components are referred to the last one third of total components in the same order as they have higher mean prices of beginning time period. We analyse, is there any information about the winner-looser separation in the initial fragments of the daily closing prices log-returns time series.The Leave-One-Out Cross-Validation with k-NN algorithm is applied on the daily log-return of components using a distance and proximity in the experiment. By looking at the error analysis, it shows that for HANGSENG and DAX index, there are clear signs of possibility to evaluate the probability of long-term success. The correlation distance matrix histograms and 2-D/3-D elastic maps generated from ViDaExpert show that the 'winner' components are closer to each other and 'winner'/'loser' components are separable on elastic maps for HANGSENG and DAX index while for the negative possibility indices, there is no sign of separation.

Spahn,
F., Vieira Neto, E., Guimarães,
A.H.F., Gorban, A.N., Brilliantov, N.V.

**A statistical model of aggregate
fragmentation**, New Journal of Physics 16, Article number 013031, 2014.

A statistical model of fragmentation of aggregates is proposed, based on the
stochastic propagation of cracks through the body. The propagation rules are
formulated on a lattice and mimic two important features of the process - a
crack moves against the stress gradient while dissipating energy during its
growth. We perform numerical simulations of the model for two-dimensional
lattice and reveal that the mass distribution for small- and intermediate-size
fragments obeys a power law, F(m)∝m^{−3/2}, in agreement
with experimental observations. We develop an analytical theory which explains
the detected power law and demonstrate that the overall fragment mass
distribution in our model agrees qualitatively with that one observed in experiments.

** **

K.I. Sofeikov, I. Romanenko, I. Tyukin, A.N. Gorban.

**Scene Analysis Assisting for AWB Using Binary
Decision Trees and Average Image Metrics.** In *Proceedings of IEEE Conference on
Consumer Electronics*, 10-13 January, Las-Vegas, USA, 2014, pp. 488-491.

We propose a technique for improving Automatic White Balance (AWB) settings in digital cameras on the basis automatic classification of image fragments in pictures. Our approach is based on constructing binary decision trees and using them as decision-making devices for identifying and locating patches of consistent texture in an image, such as grass, sky etc. We demonstrate with examples that this approach can be applied successfully to enhance color reproduction of images in challenging light conditions. Furthermore, due to low levels of false-positives, the method can be used in combination with any other AWB algorithms that do not rely on color clues obtained from the inference and analysis of content in images taken.

** **

A.N. Gorban, I. Karlin

**Hilbert's 6th Problem: exact and approximate hydrodynamic manifolds for
kinetic equations****, ***Bulletin of the American Mathematical
Society*, 51(2), 2014, 186-246. PII: S 0273-0979(2013)01439-3

The problem of the derivation of hydrodynamics from the Boltzmann equation and
related dissipative systems is formulated as the problem of a slow invariant
manifold in the space of distributions. We review a few instances where such
hydrodynamic manifolds were found analytically both as the result of summation
of the Chapman-Enskog asymptotic expansion and by the
direct solution of the invariance equation. These model cases, comprising
Grad's moment systems, both linear and nonlinear, are studied in depth in order
to gain understanding of what can be expected for the Boltzmann equation.
Particularly, the dispersive dominance and saturation of dissipation rate of
the exact hydrodynamics in the short-wave limit and the viscosity modification
at high divergence of the flow velocity are indicated as severe obstacles to
the resolution of Hilbert's 6th Problem. Furthermore, we review the derivation
of the approximate hydrodynamic manifold for the Boltzmann equation using
Newton's iteration and avoiding smallness parameters, and compare this to the
exact solutions. Additionally, we discuss the problem of projection of the
Boltzmann equation onto the approximate hydrodynamic invariant manifold using
entropy concepts. Finally, a set of hypotheses is put forward where we describe
open questions and set a horizon for what can be derived exactly or proven
about the hydrodynamic manifolds for the Boltzmann equation in the future.

** **

** **

A.N. Gorban, I. Tyukin, E. Steur, and H. Nijmeijer

**Lyapunov****-like conditions of forward invariance and
boundedness for a class of unstable systems**, *SIAM J. Control Optim*., Vol. 51, No. 3,
2013, pp. 2306-2334.

We provide Lyapunov-like characterizations of boundedness and convergence of nontrivial solutions for a class of systems with unstable invariant sets. Examples of systems to which the results may apply include interconnections of stable subsystems with one-dimensional unstable dynamics or critically stable dynamics. Systems of this type arise in problems of nonlinear output regulation, parameter estimation, and adaptive control. In addition to providing boundedness and convergence criteria, the results allow us to derive domains of initial conditions corresponding to solutions leaving a given neighborhood of the origin at least once. In contrast to other works addressing convergence issues in unstable systems, our results require neither input-output characterizations for the stable part nor estimates of convergence rates. The results are illustrated with examples, including the analysis of phase synchronization of neural oscillators with heterogeneous coupling.

E.M. Mirkes, A. Zinovyev, and A.N. Gorban,

**Geometrical
Complexity of Data Approximators**,
in I. Rojas, G. Joya, and J. Cabestany
(Eds.): IWANN 2013, Part I, *Advances in
Computation Intelligence, Springer LNCS* 7902, pp. 500–509, 2013.

There are many methods developed to approximate a cloud of vectors embedded in high-dimensional space by simpler objects: starting from principal points and linear manifolds to self-organizing maps, neural gas, elastic maps, various types of principal curves and principal trees, and so on. For each type of approximators the measure of the approximator complexity was developed too. These measures are necessary to find the balance between accuracy and complexity and to define the optimal approximations of a given type. We propose a measure of complexity (geometrical complexity) which is applicable to approximators of several types and which allows comparing data approximations of different types.

I. Tyukin, A.N. Gorban

**Explicit
Reduced-Order Integral Formulations of State and Parameter Estimation Problems
for a Class of Nonlinear Systems**. In Proceedings of the 52-th IEEE International
Conference on Decision and Control (10-13 December, 2013, Florence, Italy),
IEEE, 4284-4289.

We propose a technique for reformulation ofstate and parameter estimation problems as that of matching explicitly computable definite integrals with known kernels to data. The technique applies for a class of systems of nonlinear ordinary differential equations and is aimed to exploit parallel computational streams in order to increase speed of calculations. The idea is based on the classical adaptive observers design. It has been shown that in case the data is periodic it may be possible to reduce dimensionality of the inference problem to that of the dimension of the vector of parameters entering the right-hand side of the model nonlinearly. Performance and practical implications of the method are illustrated on a benchmark model governing dynamics of voltage in generated in barnacle giant muscle.

A.N. Gorban, G.S. Yablonsky

**Grasping
Complexity***, Computers
& Mathematics with Applications*, Volume 65, Issue 10, May 2013, 1421-1426. arXiv:1303.3855 [cs.GL]
http://arxiv.org/pdf/1303.3855

The century of complexity has come. The face of science has changed. Surprisingly, when we start asking about the essence of these changes and then critically analyse the answers, the result are mostly discouraging. Most of the answers are related to the properties that have been in the focus of scientific research already for more than a century (like non-linearity)... This paper is the editorial preface to the special issue "Grasping Complexity" of the journal "Computers and Mathematics with Applications". We analyse the change of era in science, its reasons and main changes in scientific activity and give a brief review of the papers in the issue.

A.N. Gorban

**Maxallent: Maximizers of all entropies and
uncertainty of uncertainty**, *Computers
& Mathematics with Applications*, Volume 65, Issue 10, May 2013,
1438-1456. arXiv:1212.5142 [physics.data-an]
http://arxiv.org/pdf/1212.5142

The entropy maximum approach (Maxent) was developed as a minimization of the subjective uncertainty measured by the Boltzmann–Gibbs–Shannon entropy. Many new entropies have been invented in the second half of the 20th century. Now there exists a rich choice of entropies for fitting needs. This diversity of entropies gave rise to a Maxent “anarchism”. The Maxent approach is now the conditional maximization of an appropriate entropy for the evaluation of the probability distribution when our information is partial and incomplete. The rich choice of non-classical entropies causes a new problem: which entropy is better for a given class of applications? We understand entropy as a measure of uncertainty which increases in Markov processes. In this work, we describe the most general ordering of the distribution space, with respect to which all continuous-time Markov processes are monotonic (the Markov order). For inference, this approach results in a set of conditionally “most random” distributions. Each distribution from this set is a maximizer of its own entropy. This “uncertainty of uncertainty” is unavoidable in the analysis of non-equilibrium systems. Surprisingly, the constructive description of this set of maximizers is possible. Two decomposition theorems for Markov processes provide a tool for this description.

R.A. Brownlee, J. Levesley, D. Packwood, A.N. Gorban

**Add-ons for Lattice Boltzmann
Methods: Regularization, Filtering and Limiters**, *Progress in Computational Physics*, 2013,
vol. 3, 31-52. arXiv:1110.0270 [physics.comp-ph]
http://arxiv.org/pdf/1110.0270

We describe how regularization of lattice Boltzmann methods can be achieved by modifying dissipation. Classes of techniques used to try to improve regularization of LBMs include flux limiters, enforcing the exact correct production of entropy and manipulating non-hydrodynamic modes of the system in relaxation. Each of these techniques corresponds to an additional modification of dissipation compared with the standard LBGK model. Using some standard 1D and 2D benchmarks including the shock tube and lid driven cavity, we explore the effectiveness of these classes of methods.

A. N. Gorban

**Thermodynamic Tree: The Space of
Admissible Paths**,
*SIAM J. Applied
Dynamical Systems*,
Vol. 12, No. 1 (2013), pp. 246-278. DOI: 10.1137/120866919 **arXiv**** e-print**

Is a spontaneous transition from a state x to a state y allowed by thermodynamics? Such a question arises often in chemical thermodynamics and kinetics. We ask the more formal question: is there a continuous path between these states, along which the conservation laws hold, the concentrations remain non-negative and the relevant thermodynamic potential G (Gibbs energy, for example) monotonically decreases? The obvious necessary condition, G(x)≥G(y), is not sufficient, and we construct the necessary and sufficient conditions. For example, it is impossible to overstep the equilibrium in 1-dimensional (1D) systems (with n components and n-1 conservation laws). The system cannot come from a state x to a state y if they are on the opposite sides of the equilibrium even if G(x) > G(y). We find the general multidimensional analogue of this 1D rule and constructively solve the problem of the thermodynamically admissible transitions.

We study dynamical systems, which are given in a positively invariant convex polyhedron D and have a convex Lyapunov function G. An admissible path is a continuous curve along which $G$ does not increase. For x,y from D, x≥y (x precedes y) if there exists an admissible path from x to y and x~y if x≥y and y≥x. The tree of G in D is a quotient space D/~. We provide an algorithm for the construction of this tree. In this algorithm, the restriction of G onto the 1-skeleton of D (the union of edges) is used. The problem of existence of admissible paths between states is solved constructively. The regions attainable by the admissible paths are described.

** **

Andrei Zinovyev, Nadya Morozova, Alexander N. Gorban, and Annick Harel-Belan

**Mathematical
Modeling of
microRNA-Mediated Mechanisms of Translation Repression**,
in U. Schmitz et al. (eds.), *MicroRNA
Cancer Regulation: Advanced Concepts, Bioinformatics and Systems Biology Tools,
Advances in Experimental Medicine and Biology *Vol. 774, Springer, 2013, pp.
189-224.

MicroRNAs can affect the protein translation using nine mechanistically different mechanisms, including repression of initiation and degradation of the transcript. There is a hot debate in the current literature about which mechanism and in which situations has a dominant role in living cells. The worst, same experimental systems dealing with the same pairs of mRNA and miRNA can provide ambiguous evidences about which is the actual mechanism of translation repression observed in the experiment. We start with reviewing the current knowledge of various mechanisms of miRNA action and suggest that mathematical modeling can help resolving some of the controversial interpretations. We describe three simple

mathematical models of miRNA translation that can be used as tools in interpreting the experimental data on the dynamics of protein synthesis. The most complex model developed by us includes all known mechanisms of miRNA action. It allowed us to study possible dynamical patterns corresponding to different miRNA-mediated mechanisms of translation repression and to suggest concrete recipes on determining the dominant mechanism of miRNA action in the form of kinetic signatures. Using computational experiments and systematizing existing evidences from the literature, we justify a hypothesis about co-existence of distinct miRNA-mediated mechanisms of translation repression. The actually observed mechanism will be that acting on or changing the sensitive parameters of the translation process. The limiting place can vary from one experimental setting to another. This model explains the majority of existing controversies reported.

A.N. Gorban, E.M. Mirkes, G.S. Yablonsky

**Thermodynamics in the limit of
irreversible reactions**, *Physica** A* 392 (2013) 1318–1335.

For many complex real physicochemical systems, the detailed mechanism
includes both reversible and irreversible reactions. Such systems are typical
in homogeneous combustion and heterogeneous catalytic oxidation. Most complex
enzyme reactions include irreversible steps. Classical thermodynamics has no
limit for irreversible reactions, whereas kinetic equations may have such a
limit. We represent systems with irreversible reactions as the limits of fully
reversible systems when some of the equilibrium concentrations tend to zero.
The structure of the limit reaction system crucially depends on the relative
rates of this tendency to zero. We study the dynamics of the limit system and
describe its limit behavior as t → ∞. If the
reversible systems obey the principle of detailed balance then the limit system
with some irreversible reactions must satisfy the extended principle of
detailed balance. It is formulated and proven in the form of two conditions:

(i) the reversible part satisfies the principle of
detailed balance and

(ii) the convex hull of the stoichiometric vectors of the irreversible
reactions does not intersect the linear span of the stoichiometric vectors of
the reversible reactions.

These conditions imply the existence of the global Lyapunov
functionals and allow an algebraic description of the
limit behavior. Thermodynamic theory of the irreversible
limit of reversible reactions is illustrated by the analysis of hydrogen
combustion.

Alexander N. Gorban,

**Local
equivalence of reversible and general Markov kinetics,** *Physica** A *392 (2013)
1111–1121.

We consider continuous-time Markov kinetics with a finite number of
states and a positive equilibrium P^{∗}.
This class of systems is significantly wider than the systems with detailed
balance. Nevertheless, we demonstrate that for an arbitrary probability distribution P and a
general system there exists a system with detailed balance and the same
equilibrium that has the same velocity dP/dt at point P. The results are extended to nonlinear
systems with the generalized mass action law.

** **

** **

Alexander N. Gorban and Dave Packwood, **Allowed and forbidden
regimes of entropy balance in lattice Boltzmann collisions****,
**Physical Review E **86**, 025701(R) (2012).

We study the possibility of modifying collisions in the lattice Boltzmann
method to keep the proper entropy balance. We demonstrate that in the space of
distributions operated on by lattice Boltzmann methods which respect a
Boltzmann type *H *theorem, there exists a
vicinity of the equilibrium where collisions with entropy balance are possible
and, at the same time, there exists a region of nonequilibrium
distributions where such collisions are impossible. In particular, for a
strictly concave and uniformly bounded entropy function with positive
equilibria, we show that proper entropy balance is always possible sufficiently
close to the local equilibrium and it is impossible sufficiently far from it,
where additional dissipation has to appear.We also
present some nonclassical entropies that do not share
this concern. The cases where the distribution enters the region far from
equilibrium typically occur in flows with low viscosity and/or high Mach number
flows and in simulations on coarse grids

** **

Nadya Morozova, Andrei Zinovyev, Nora Nonne, Linda-Louise Pritchard, Alexander N. Gorban, and Annick Harel-Bellan,

**Kinetic signatures of
microRNA modes of action****,** *RNA*, Vol. 18, No. 9 (2012) 1635-1655, doi:10.1261/rna.032284.112

MicroRNAs (miRNAs) are key regulators of all important biological processes, including development, differentiation, and cancer. Although remarkable progress has been made in deciphering the mechanisms used by miRNAs to regulate translation, many contradictory findings have been published that stimulate active debate in this field. Here we contribute to this discussion in three ways. First, based on a comprehensive analysis of the existing literature, we hypothesize a model in which all proposed mechanisms of microRNA action coexist, and where the apparent mechanism that is detected in a given experiment is determined by the relative values of the intrinsic characteristics of the target mRNAs and associated biological processes. Among several coexisting miRNA mechanisms, the one that will effectively be measurable is that which acts on or changes the sensitive parameters of the translation process. Second, we have created a mathematical model that combines nine known mechanisms of miRNA action and estimated the model parameters from the literature. Third, based on the mathematical modeling, we have developed a computational tool for discriminating among different possible individual mechanisms of miRNA action based on translation kinetics data that can be experimentally measured (kinetic signatures). To confirm the discriminatory power of these kinetic signatures and to test our hypothesis, we have performed several computational experiments with the model in which we simulated the coexistence of several miRNA action mechanisms in the context of variable parameter values of the translation.

Ovidiu Radulescu, Alexander N. Gorban, Andrei Zinovyev, Vincent Noel

**Reduction of dynamical biochemical reaction networks in computational
biology**, Frontiers
in Genetics (Bioinformatics
and Computational Biology). July2012, Volume3, Article 131. (e-print arXiv:1205.2851
[q-bio.MN])

Biochemical networks are used in
computational biology, to model mechanistic details of systems involved in cell
signaling, metabolism, and regulation of gene
expression. Parametric and structural uncertainty, as well as combinatorial
explosion are strong obstacles against analyzing the
dynamics of large models of this type. Multiscaleness,
an important property of these networks, can be used to get past some of these
obstacles. Networks with many well separated time scales, can be reduced to
simpler models, in a way that depends only on the orders of magnitude and not
on the exact values of the kinetic parameters. The main idea used for such
robust simplifications of networks is the concept of dominance among model elements,
allowing hierarchical organization of these elements according to their effects
on the network dynamics. This concept finds a natural formulation in tropical
geometry. We revisit, in the light of these new ideas, the main approaches to
model reduction of reaction networks, such as quasi-steady state (QSS) and
quasi-equilibrium approximations (QE), and provide practical recipes for model
reduction of linear and non-linear networks. We also discuss the application of
model reduction to the problem of parameter identification, via backward
pruning machine learning techniques.

**Local Equivalence of Reversible and General Markov Kinetics****,** __arXiv:1205.2052__ [physics.chem-ph]

We consider continuous--time Markov kinetics with a finite number of states
and a given positive equilibrium distribution *P**. For an arbitrary probability distribution *P* we study the possible right hand sides, d*P*/d*t*, of the Kolmogorov (master)
equations. We describe the cone of possible values of the velocity, d*P*/d*t*, as a function
of *P* and *P**. We prove that, surprisingly, these cones coincide for the class
of all Markov processes with equilibrium *P**
and for the reversible Markov processes with detailed balance at this
equilibrium. Therefore, for an arbitrary probability distribution *P* and a general system there exists a
system with detailed balance and the same equilibrium that has the same
velocity d*P*/d*t* at point *P*. The set of Lyapunov
functions for the reversible Markov processes coincides with the set of Lyapunov functions for general Markov kinetics. The results
are extended to nonlinear systems with the generalized mass action law.

Alexander N. Gorban, Andrei Zinovyev, Nadya Morozova, Annick Harel-Bellan

**Modeling coupled transcription, translation and degradation and miRNA-based
regulation of this process**, arXiv:1204.5941
[q-bio.MN]

The translation-transcription process with the description of the most basic
"elementary" processes consists in: 1) production of mRNA molecules,
2) initiation of these molecules by circularization with help of initiation
factors, 3) initiation of translation, recruiting the small ribosomal subunit,
4) assembly of full ribosomes, 5) elongation, i.e. movement of ribosomes along
mRNA with production of protein, 6) termination of translation, 7) degradation
of mRNA molecules. A certain complexity in the mathematical formulation of this
process arises when one tries to take into account the phenomenon of polysome first, when several ribosomes are producing peptides
on a single mRNA at the same time. This leads to multiplicity of possible
states of mRNA with various numbers of ribosomes with potentially different
dynamics, interaction between ribosomes and other difficulties. In this
preprint we provide 1) detailed mechanistic description of the translation
process with explicit representation of every state of translating mRNA,
followed by 2) deriving the simplest and basic ODE model of coupled
transcription, translation and degradation, and 3) developing a model suitable
for describing all known mechanisms of miRNA action on translation. The basic
model is constructed by correct lumping of the detailed model states and by
separating the description of ribosomal turnover. It remains linear under
assumption of that the translation is not limited by availability of ribosomal
subunits or initiation factors. The only serious limitation of this type of
translation modeling is in that it does not take into account possible
interactions between ribosomes. The latter might lead to more complex phenomena
which can be taken into account in simulatory models
of the detailed representation of translation at the cost of more difficult
analytical analysis of the model.

A. Zinovyev, N. Morozova, A. N. Gorban, A. Harel-Belan

**Mathematical modeling of microRNA-mediated mechanisms of translation
repression****,** arXiv:1202.1243 [q-bio.MN]

MicroRNAs can affect the protein translation using nine mechanistically
different mechanisms, including repression of initiation and degradation of the
transcript. There is a hot debate in the current literature about which
mechanism and in which situations has a dominant role in living cells. The
worst, same experimental systems dealing with the same pairs of mRNA and miRNA
can provide ambiguous evidences about which is the actual mechanism of
translation repression observed in the experiment. We start with reviewing the
current knowledge of various mechanisms of miRNA action and suggest that
mathematical modeling can help resolving some of the controversial
interpretations. We describe three simple mathematical models of miRNA
translation that can be used as tools in interpreting the experimental data on
the dynamics of protein synthesis. The most complex model developed by us
includes all known mechanisms of miRNA action. It allowed us to study possible
dynamical patterns corresponding to different miRNA-mediated mechanisms of
translation repression and to suggest concrete recipes on determining the
dominant mechanism of miRNA action in the form of kinetic signatures. Using
computational experiments and systematizing existing evidences from the
literature, we justify a hypothesis about co-existence of distinct
miRNA-mediated mechanisms of translation repression. The actually observed
mechanism will be that acting on or changing the limiting "place" of
the translation process. The limiting place can vary from one experimental
setting to another. This model explains the majority of existing controversies
reported.

**Thermodynamic Tree: The Space of Admissible Paths****,** arXiv:1201.6315 [cond-mat.stat-mech]

Is a spontaneous transition from a state x to a
state y allowed by thermodynamics? Such a question arises often in chemical
thermodynamics and kinetics. We ask the more formal question: is there a
continuous path between these states, along which the conservation laws hold,
the concentrations remain non-negative and the relevant thermodynamic potential
*G* (Gibbs energy, for example)
monotonically decreases? The obvious necessary condition, *G*(*x*)≥*G*(*y*),
is not sufficient, and we construct the necessary and sufficient conditions.
For example, it is impossible to overstep the equilibrium in 1-dimensional (1D)
systems (with n components and n-1 conservation laws). The system cannot come
from a state x to a state y if they are on the opposite sides of the
equilibrium even if *G*(*x*) >*G*(*y*). We find the general
multidimensional analogue of this 1D rule and constructively solve the problem
of the thermodynamically admissible transitions.

We study dynamical systems, which are given in a positively invariant convex
polyhedron and have a convex Lyapunov function *G*. An admissible path is a continuous
curve along which *G* does not
increase. For *x,y* from *D,
x>y *(*x* precedes* y*) if there exists an admissible path
from *x* to *y* and *x~y*
if *x>y* and *y>x*. The tree of *G* in *D* is a quotient space *D/~*. We provide an algorithm for the
construction of this tree. In this algorithm, the restriction of *G* onto the 1-skeleton of *D* (the union of edges) is used. The
problem of existence of admissible paths between states is solved
constructively. The regions attainable by the admissible paths are described.

** **

** **

**2011**

A.N. Gorban, G.S.Yablonsky

**Extended
detailed balance for systems with irreversible reactions**,* Chemical Engineering Science* 66 (2011)
5388–5399.

The principle of detailed balance states that in equilibrium each elementary process is equilibrated by its reverse process. For many real physico-chemical complex systems (e.g. homogeneous combustion, heterogeneous catalytic oxidation, most enzyme reactions etc), detailed mechanisms include both reversible and irreversible reactions. In this case, the principle of detailed balance cannot be applied directly. We represent irreversible reactions as limits of reversible steps and obtain the principle of detailed balance for complex mechanisms with some irreversible elementary processes. We proved two consequences of the detailed balance for these mechanisms: the structural condition and the algebraic condition that form together the extended form of detailed balance. The algebraic condition is the principle of detailed balance for the reversible part. The structural condition is: the convex hull of the stoichiometric vectors of the irreversible reactions has empty intersection with the linear span of the stoichiometric vectors of the reversible reaction. Physically, this means that the irreversible reactions cannot be included in oriented pathways.

The systems with the extended form of detailed balance are also the limits of the reversible systems with detailed balance when some of the equilibrium concentrations (or activities) tend to zero. Surprisingly, the structure of the limit reaction mechanism crucially depends on the relative speeds of this tendency to zero.

A. N. Gorban, D.
Packwood

**Possibility and Impossibility of the Entropy
Balance in Lattice Boltzmann Collisions****,** arXiv:1111.5994 [physics.comp-ph]

We demonstrate that in the space of distributions operated on by lattice
Boltzmann methods that there exists a vicinity of the equilibrium where
collisions with entropy balance are possible and, at the same time, there exist
an area of nonequilibrium distributions where such
collisions are impossible. We calculate and graphically represent these areas
for some simple entropic equilibria using single relaxation time models.
Therefore it is shown that the definition of an entropic LBM is incomplete
without a strategy to deal with certain highly nonequilibrium
states. Such strategies should be explicitly stated as they may result in the
production of additional entropy.

R.A. Brownlee, J. Levesley, D. Packwood, A.N. Gorban

**Add-ons for Lattice Boltzmann Methods: Regularization, Filtering and
Limiters****, **arXiv:1110.0270 [physics.comp-ph]**
**We describe how regularization of lattice
Boltzmann methods can be achieved by modifying dissipation. Classes of
techniques used to try to improve regularization of LBMs include flux limiters,
enforcing the exact correct production of entropy and manipulating
non-hydrodynamic modes of the system in relaxation. Each of these techniques
corresponds to an additional modification of dissipation compared with the
standard LBGK model. Using some standard 1D and 2D benchmarks including the
shock tube and lid driven cavity, we explore the effectiveness of these classes
of methods.

E. Chiavazzo, IV Karlin,
AN Gorban, and K Boulouchos.

**Efficient simulations of detailed combustion fields via the lattice Boltzmann
method.**
International Journal of Numerical Methods for Heat & Fluid Flow 21, no. 5
(2011), 494-517.

Purpose

– The paper aims to be a first step
toward the efficient, yet accurate, solution of detailed combustion fields
using the lattice Boltzmann (LB) method, where applications are still limited
due to both the stiffness of the governing equations and the large amount of
fields to solve.

Design/methodology/approach

– The suggested methodology for model
reduction is developed in the setting of slow invariant manifold construction,
including details of the while. The simplest LB equation is used in order to
work out the procedure of coupling of the reduced model with the flow solver.

Findings

– The proposed method is validated with
the 2D simulation of a premixed laminar flame in the hydrogen‐air mixture,
where a remarkable computational speedup and memory saving are demonstrated.

Research limitations/implications

– Because of the chosen detailed LB
model, the flow field may be described with unsatisfactory accuracy: this
motivates further investigation in this direction in the near future.

Practical implications

– A new framework of simulation of
reactive flows is available, based on a coupling between accurate reduced
reaction mechanism and the LB representation of the flow phenomena. Hence, the
paper includes implications on how to perform accurate reactive flow
simulations at a fraction of the cost required in the detailed model.

Originality/value

– This paper meets an increasing need to
have efficient and accurate numerical tools for modelling complex phenomena,
such as pollutant formation during combustion.

F. Spahn, E. V. Neto, A. H.
F. Guimaraes, A. N.
Gorban, N. V. Brilliantov

**A Statistical Model of Aggregates Fragmentation****, **arXiv:1106.2721 [cond-mat.stat-mech]

A statistical model of fragmentation of aggregates is proposed, based on
the stochastic propagation of cracks through the body. The propagation rules
are formulated on a lattice and mimic two important features of the process --
a crack moves against the stress gradient and its energy depletes as it grows.
We perform numerical simulations of the model for two-dimensional lattice and
reveal that the mass distribution for small and intermediate-size fragments
obeys a power-law, F(m)\propto
m^(-3/2), in agreement with experimental observations. We develop an analytical
theory which explains the detected power-law and demonstrate that the overall
fragment mass distribution in our model agrees qualitatively with that,
observed in experiments.

A.N.
Gorban, H.P. Sargsyan and H.A. Wahab

**Quasichemical**** Models of Multicomponent Nonlinear
Diffusion****,** *Mathematical Modelling of Natural Phenomena*,
Volume 6 /
Issue 05,
(2011), 184−262.

Diffusion preserves the positivity of concentrations, therefore, multicomponent
diffusion should be nonlinear if there exist non-diagonal terms. The vast
variety of nonlinear multicomponent diffusion equations should be ordered and
special tools are needed to provide the systematic construction of the
nonlinear diffusion equations for multicomponent mixtures with significant
interaction between components. We develop an approach to nonlinear
multicomponent diffusion based on the idea of the reaction mechanism borrowed
from chemical kinetics.

Chemical kinetics gave rise to very seminal tools for the modeling of processes. This is the stoichiometric algebra supplemented by the simple kinetic law. The results of this invention are now applied in many areas of science, from particle physics to sociology. In our work we extend the area of applications onto nonlinear multicomponent diffusion.

We demonstrate, how the mechanism based approach to multicomponent diffusion can be included into the general thermodynamic framework, and prove the corresponding dissipation inequalities. To satisfy thermodynamic restrictions, the kinetic law of an elementary process cannot have an arbitrary form. For the general kinetic law (the generalized Mass Action Law), additional conditions are proved. The cell–jump formalism gives an intuitively clear representation of the elementary transport processes and, at the same time, produces kinetic finite elements, a tool for numerical simulation

A. Gorban and S. Petrovskii

**Collective dynamics: when one plus one does
not make two****, ***Mathematical Medicine and Biology *(2011) 28, 85−88.

A brief introduction into the interdisciplinary field of collective dynamics is given, followed by an overview of ‘Mathematical Models of Collective Dynamics in Biology and Evolution’ (University of Leicester, 11–13 May 2009). Collective dynamics—understood as the dynamics arising from the interplay between the constituting elementary argents or parts of a more complex system—has been one of the main paradigms of the natural sciences over the last several decades.

A.N. Gorban and M. Shahzad

**The Michaelis-Menten-Stueckelberg
Theorem****.** *Entropy* **2011**,
*13*, 966-1019.

We
study chemical reactions with complex mechanisms under two assumptions: (i) intermediates are present in small amounts (this is the
quasi-steady-state hypothesis or QSS) and (ii) they are in equilibrium
relations with substrates (this is the quasiequilibrium
hypothesis or QE). Under these assumptions, we prove the generalized mass
action law together with the basic relations between kinetic factors, which are
sufficient for the positivity of the entropy production but hold even without microreversibility, when the detailed balance is not
applicable. Even though QE and QSS produce useful approximations by themselves,
only the combination of these assumptions can render the possibility beyond the
“rarefied gas” limit or the “molecular chaos” hypotheses. We do not use any a
priori form of the kinetic law for the chemical reactions and describe their
equilibria by thermodynamic relations. The transformations of the intermediate
compounds can be described by the Markov kinetics because of their low density
(*low density of elementary events*). This combination of assumptions was
introduced by Michaelis and Menten
in 1913. In 1952, Stueckelberg used the same
assumptions for the gas kinetics and produced the remarkable semi-detailed
balance relations between collision rates in the Boltzmann equation that are
weaker than the detailed balance conditions but are still sufficient for the
Boltzmann *H*-theorem to be valid. Our results are obtained within the Michaelis-Menten-Stueckelbeg conceptual framework.

G. S. Yablonsky, A. N.
Gorban, D. Constales, V. V. Galvita
and G. B. Marin

**Reciprocal
relations between kinetic curves,**** ***EPL,* 93 (2011) 20004.

We study coupled irreversible processes. For
linear or linearized kinetics with microreversibility, ,
the kinetic operator *K* is symmetric in the entropic inner product. This
form of Onsager's reciprocal relations implies that the shift in time, exp(*Kt*), is also a symmetric operator. This generates
the reciprocity relations between the kinetic curves. For example, for the
Master equation, if we start the process from the *i*-th
pure state and measure the probability *p _{j}*(

A.N. Gorban, D. Roose

**Preface****, **In: Coping
with Complexity: Model Reduction and Data Analysis, A.N. Gorban and D. Roose (eds.), Lecture Notes in Computational Science and
Engineering, 75, Springer: Heidelberg – Dordrecht - London -New York, 2011, pp.
V-VI.

A mathematical model is an intellectual device that works. …

A.N. Gorban

**Self-simplification in Darwin’s Systems,****
**In: Coping with Complexity: Model Reduction and Data Analysis, A.N. Gorban
and D. Roose (eds.), Lecture Notes in Computational
Science and Engineering, 75, Springer: Heidelberg – Dordrecht - London -New
York, 2011, pp. 311-344

We prove that a non-linear kinetic system with *conservation
of supports *for distributions has generically limit distributions with
final support only. The conservation of support has a biological
interpretation: *inheritance*. We call systems with inheritance “Darwin’s
systems”. Such systems are apparent in many areas of biology, physics (the
theory of parametric wave interaction), chemistry and economics. The finite
dimension of limit distributions demonstrates effects of *natural selection*.
Estimations of the asymptotic dimension are presented. After some initial time,
solution of a kinetic equation with conservation of support becomes a finite
set of narrow peaks that become increasingly narrow over time and move
increasingly slowly. It is possible that these peaks do not tend to fixed positions,
and the path covered tends to infinity as *t **→
∞*. The *drift equations *for peak motion are obtained. They
describe the asymptotic layer near the *omega*-limit distributions with
finite support .

D.J. Packwood, J. Levesley, and A.N. Gorban

**Time step
expansions and the invariant manifold approach to lattice Boltzmann models**,
In: Coping with Complexity: Model Reduction and Data Analysis, A.N. Gorban and
D. Roose (eds.), Lecture Notes in Computational
Science and Engineering, 75, Springer: Heidelberg – Dordrecht - London -New
York, 2011, pp. 169-206.

The classical method for deriving the macroscopic dynamics of a lattice Boltzmann system is to use a combination of different approximations and expansions. Usually a Chapman-Enskog analysis is performed, either on the continuous Boltzmann system, or its discrete velocity counterpart. Separately a discrete time approximation is introduced to the discrete velocity Boltzmann system, to achieve a practically useful approximation to the continuous system, for use in computation. Thereafter, with some additional arguments, the dynamics of the Chapman-Enskog expansion are linked to the discrete time system to produce the dynamics of the completely discrete scheme. In this paper we put forward a different route to the macroscopic dynamics. We begin with the system discrete in both velocity space and time. We hypothesize that the alternating steps of advection and relaxation, common to all lattice Boltzmann schemes, give rise to a slow invariant manifold. We perform a time step expansion of the discrete time dynamics using the invariance of the manifold. Finally we calculate the dynamics arising from this system. By choosing the fully discrete scheme as a starting point we avoid mixing approximations and arrive at a general form of the microscopic dynamics up to the second order in the time step. We calculate the macroscopic dynamics of two commonly used lattice schemes up to the first order, and hence find the precise form of the deviation from the Navier-Stokes equations in the dissipative term, arising from the discretization of velocity space.

Finally we perform a short wave perturbation on the dynamics of these example systems, to find the necessary conditions for their stability.

A.N. Gorban

**Kinetic
path summation, multi-sheeted extension of master equation, and evaluation of
ergodicity coefficient**, *Physica** A* 390 (2011)
1009–1025.

We study the master equation with time-dependent coefficients, a linear kinetic equation for the Markov chains or for the monomolecular chemical kinetics. For the solution of this equation a path summation formula is proved. This formula represents the solution as a sum of solutions for simple kinetic schemes (kinetic paths), which are available in explicit analytical form. The relaxation rate is studied and a family of estimates for the relaxation time and the ergodicity coefficient is developed. To calculate the estimates we introduce the multi-sheeted extensions of the initial kinetics. This approach allows us to exploit the internal (‘‘micro’’) structure of the extended kinetics without perturbation of the base kinetics.

A.N. Gorban, L.I. Pokidysheva,·E,V. Smirnova, T.A. Tyukina.

**Law of the Minimum Paradoxes****, ***Bull Math Biol*
73(9) (2011), 2013-2044; Online first
19.11.2010,

The “Law of the
Minimum” states that growth is controlled by the scarcest resource (limiting
factor). This concept was originally applied to plant or crop growth (Justus
von Liebig, 1840) and quantitatively supported by many experiments. Some
generalizations based on more complicated “dose-response” curves were proposed.
Violations of this law in natural and experimental ecosystems were also reported.
We study models of adaptation in ensembles of similar organisms under load of
environmental factors and prove that violation of Liebig’s law follows from
adaptation effects. If the fitness of an organism in a fixed environment
satisfies the Law of the Minimum then adaptation equalizes the pressure of
essential factors and, therefore, acts against the Liebig’s law. This is the *the** Law of the Minimum paradox*: if for a
randomly chosen pair “organism–environment” the Law of the Minimum typically
holds, then in a well-adapted system, we have to expect violations of this law.

For the opposite interaction of factors (a synergistic system of factors which amplify each other), adaptation leads from factor equivalence to limitations by a smaller number of factors.

For analysis of adaptation, we develop a system of models based on Selye’s idea of the universal adaptation resource (adaptation energy). These models predict that under the load of an environmental factor a population separates into two groups (phases): a less correlated, well adapted group and a highly correlated group with a larger variance of attributes, which experiences problems with adaptation. Some empirical data are presented and evidences of interdisciplinary applications to econometrics are discussed.

A.N.** **Gorban,
E.V. Smirnova, T.A. Tyukina,

**Correlations,
risk and crisis: From physiology to finance****,**
*Physica** A*, Vol. 389, Issue 16, 2010,
3193-3217. **Number
9 in the Top Hottest Articles in the Journal, April to June 2010**

We study the dynamics of correlation and variance in systems under the load of environmental factors. A universal effect in ensembles of similar systems under the load of similar factors is described: in crisis, typically, even before obvious symptoms of crisis appear, correlation increases, and, at the same time, variance (and volatility) increases too. This effect is supported by many experiments and observations of groups of humans, mice, trees, grassy plants, and on financial time series.

A general approach to the explanation of the effect through dynamics of individual adaptation of similar non-interactive individuals to a similar system of external factors is developed. Qualitatively, this approach follows Selye’s idea about adaptation energy.

A.N.** **Gorban

We study the Master equation with time--dependent coefficients, a linear kinetic equation for the Markov chains or for the monomolecular chemical kinetics. For the solution of this equation a paths summation formula is proved. This formula represents the solution as a sum of solutions for simple kinetic schemes (kinetic paths), which are available in explicit analytical form. The relaxation rate is studied and a family of estimates for the relaxation time and the ergodicity coefficient is developed. To calculate the estimates we introduce the multi--sheeted extensions} of the initial kinetics. This approach allows us to exploit the internal ("micro")structure of the extended kinetics without perturbation of the base kinetics.

Ovidiu Radulescu, Alexander N. Gorban, Andrei
Zinovyev,

**Pruning, pooling and limiting steps in
metabolic networks****, **Modelling Complex Biological Systems,
Proceedings of The Évry Spring School, May 3^{rd} - 7^{th}, 2010, Edited by Patrick
Amar, Franҫois Képès, Vic
Norris, EDP Sciences, Évry,
2010, pp. 109-126.

Dynamics of metabolic systems can be modelled by systems of differential equations. Realistic models of metabolism allowing to integrate genome scale data should have very large size and thus face problems related to incompleteness of the information on their structure and parameters. We discuss how model reduction techniques that use qualitative information on the order of magnitude of parameters can be applied to simplify large models of differential equations.

A. N. Gorban, A. Zinovyev.

**Principal manifolds and graphs in practice:
from molecular biology to dynamical systems****, ***International Journal of Neural Systems,*
Vol. 20, No. 3 (2010) 219–232.

We present several applications of non-linear data modeling, using principal manifolds and principal graphs constructed using the metaphor of elasticity (elastic principal graph approach). These approaches are generalizations of the Kohonen’s self-organizing maps, a class of artificial neural networks. On several examples we show advantages of using non-linear objects for data approximation in comparison to the linear ones. We propose four numerical criteria for comparing linear and non-linear mappings of datasets into the spaces of lower dimension. The examples are taken from comparative political science, from analysis of high-throughput data in molecular biology, from analysis of dynamical systems.

E. Chiavazzo, I.V. Karlin, A.N. Gorban, K. Boulouchos,

**Coupling of the model reduction technique
with the lattice Boltzmann method**, Combustion
and Flame 157 (2010) 1833–1849 doi:10.1016/j.combustflame.2010.06.009

A new framework of simulation of reactive flows is proposed based on a coupling between accurate reduced reaction mechanism and the lattice Boltzmann representation of the flow phenomena. The model reduction is developed in the setting of slow invariant manifold construction, and the simplest lattice Boltzmann equation is used in order to work out the procedure of coupling of the reduced model with the flow solver. Practical details of constructing slow invariant manifolds of a reaction system under various thermodynamic conditions are reported. The proposed method is validated with the two-dimensional simulation of a premixed counterflow flame in the hydrogen-air mixture.

Gorban A.N., Gorban P.A., Judge G.

**Entropy: The Markov Ordering Approach**. *Entropy*.
2010; 12(5):1145-1193. **NEW** **“ Entropy Best Paper Award” for 2014.** GorbanGorbanJudgeEntropy2010.pdf

The
focus of this article is on entropy and Markov processes. We study the
properties of functionals which are invariant with
respect to monotonic transformations and analyze two invariant “additivity”
properties: (i) existence of a monotonic
transformation which makes the functional additive with respect to the joining
of independent systems and (ii) existence of a monotonic transformation which
makes the functional additive with respect to the partitioning of the space of
states. All Lyapunov functionals
for Markov chains which have properties (i) and (ii)
are derived. We describe the most general ordering of the distribution space,
with respect to which all continuous-time Markov processes are monotonic (the *Markov
order*). The solution differs significantly from the
ordering given by the inequality of entropy growth. For inference, this
approach results in a convex compact set of conditionally “most random”
distributions.

A. N. Gorban and V. M. Cheresiz,

**Slow Relaxations and Bifurcations of the Limit
Sets of Dynamical Systems. I. Bifurcations of Limit Sets,****
***Journal of Applied and Industrial Mathematics**,
*2010, Vol. 4, No. 1, pp. 54–64.

We consider one-parameter semigroups of homeomorphisms depending continuously on the parameters. We study the phenomenon of slow relaxation that consists in anomalously slow motion to the limit sets. We investigate the connection between slow relaxations and bifurcations of limit sets and other singularities of the dynamics. The statements of some of the problems stem from mathematical chemistry.

A. N. Gorban and V. M. Cheresiz,

**Slow Relaxations and Bifurcations of the Limit
Sets of Dynamical Systems. II. Slow Relaxations of a Family of Semiflows,**** ***Journal of Applied and Industrial Mathematics**,
*2010, Vol. 4, No. 2, pp. 182–190.

We propose a number of approaches to the notion of the relaxation time of a dynamical system which are motivated by the problems of chemical kinetics, give exact mathematical definitions of slow relaxations, study their possible reasons, among which an important role is played by bifurcations of limit sets.

E. Chiavazzo, I.V. Karlin, and A.N. Gorban,

**The Role of Thermodynamics in Model Reduction
when Using Invariant Grids****, ***Commun**. Comput. Phys.,* Vol. **8**, No. 4 (2010), pp.
701-734.

In the present work, we develop in detail the process leading to reduction of models in chemical kinetics when using the Method of Invariant Grids (MIG). To this end, reduced models (invariant grids) are obtained by refining initial approximations of slow invariant manifolds, and used for integrating smaller and less stiff systems of equations capable to recover the detailed description with high accuracy. Moreover, we clarify the role played by thermodynamics in model reduction, and carry out a comparison between detailed and reduced solutions for a model hydrogen oxidation reaction.

Andrei Zinovyev, Nadya Morozova, Nora Nonne, Emmanuel
Barillot, Annick Harel-Bellan,
Alexander N Gorban

**Dynamical modeling
of microRNA action on the protein translation process,** * **BMC Systems Biology* 2010, 4:13 (24 February 2010)

**Background**

Protein translation is a multistep process which can be represented as a cascade of biochemical reactions (initiation, ribosome assembly, elongation, etc.), the rate of which can be regulated by small non-coding microRNAs through multiple mechanisms. It remains unclear what mechanisms of microRNA action are the most dominant: moreover, many experimental reports deliver controversal messages on what is the concrete mechanism actually observed in the experiment. Nissan and Parker have recently demonstrated that it might be impossible to distinguish alternative biological hypotheses using the steady state data on the rate of protein synthesis. For their analysis they used two simple kinetic models of protein translation.

**Results**

In contrary to the study by Nissan and Parker, we show that dynamical data allow to discriminate some of the mechanisms of microRNA action. We demonstrate this using the same models as developed by Nissan and Parker for the sake of comparison but the methods developed (asymptotology of biochemical networks) can be used for other models. We formulate a hypothesis that the effect of microRNA action is measurable and observable only if it affects the dominant system (generalization of the limiting step notion for complex networks) of the protein translation machinery. The dominant system can vary in different experimental conditions that can partially explain the existing controversy of some of the experimental data.

**Conclusions**

Our analysis of the transient protein translation dynamics shows that it gives enough information to verify or reject a hypothesis about a particular molecular mechanism of microRNA action on protein translation. For multiscale systems only that action of microRNA is distinguishable which affects the parameters of dominant system (critical parameters), or changes the dominant system itself. Dominant systems generalize and further develop the old and very popular idea of limiting step. Algorithms for identifying dominant systems in multiscale kinetic models are straightforward but not trivial and depend only on the ordering of the model parameters but not on their concrete values. Asymptotic approach to kinetic models allows to put in order diverse experimental observations in complex situations when many alternative hypotheses co-exist.

A. N.
Gorban, O.
Radulescu, A. Y.
Zinovyev,

**Asymptotology**** of
chemical reaction networks,** Chemical Engineering Science 65 (2010) 2310–2324
GorbRadZinCES2010Rev.pdf

The concept of the limiting step is extended to the asymptotology of multiscale reaction networks. Complete theory for linear networks with well separated reaction rate constants is developed. We present algorithms for explicit approximations of eigenvalues and eigenvectors of kinetic matrix. Accuracy of estimates is proven. Performance of the algorithms is demonstrated on simple examples. Application of algorithms to nonlinear systems is discussed.

A.N.
Gorban, E.V. Smirnova, T.A. Tyukina

**General**** Laws of Adaptation to Environmental
Factors: from Ecological Stress to Financial Crisis.**** **Math.
Model. Nat. Phenom. Vol. 4, No. 6, 2009, pp. 1-53

We study ensembles of similar systems under load of environmental factors. The phenomenon of adaptation has similar properties for systems of different nature. Typically, when the load increases above some threshold, then the adapting systems become more different (variance increases), but the correlation increases too. If the stress continues to increase then the second threshold appears: the correlation achieves maximal value, and start to decrease, but the variance continue to increase. In many applications this second threshold is a signal of approaching of fatal outcome. This effect is supported by many experiments and observation of groups of humans, mice, trees, grassy plants, and on financial time series.

A general approach to explanation of the effect
through dynamics of adaptation is developed. H. Selye
introduced “adaptation energy” for explanation of adaptation phenomena. We
formalize this approach in *factors – resource *models and develop
hierarchy of models of adaptation. Different organization of interaction
between factors (Liebig’s versus synergistic systems) lead to different
adaptation dynamics. This gives an explanation to qualitatively different
dynamics of correlation under different types of load and to some deviation
from the typical reaction to stress. In addition to the “quasistatic”
optimization factor – resource models, dynamical models of adaptation are
developed, and a simple model (three variables) for adaptation to one factor
load is formulated explicitly.

A. N.
Gorban, A. Y.
Zinovyev

**Principal Graphs and Manifolds,**
Chapter 2 in: Handbook of Research on Machine Learning Applications and Trends:
Algorithms, Methods, and Techniques, Emilio Soria Olivas et al. (eds), IGI Global, Hershey, PA,
USA, 2009, pp. 28-59.

In many physical, statistical, biological and other investigations it is desirable to approximate a system of points by objects of lower dimension and/or complexity. For this purpose, Karl Pearson invented principal component analysis in 1901 and found ‘lines and planes of closest fit to system of points’. The famous k-means algorithm solves the approximation problem too, but by finite sets instead of lines and planes. This chapter gives a brief practical introduction into the methods of construction of general principal objects (i.e., objects embedded in the ‘middle’ of the multidimensional data set). As a basis, the unifying framework of mean squared distance approximation of finite datasets is selected. Principal graphs and manifolds are constructed as generalisations of principal components and k-means principal points. For this purpose, the family of expectation/maximisation algorithms with nearest generalisations is presented. Construction of principal graphs with controlled complexity is based on the graph grammar approach.

A.N.
Gorban, L.I.
Pokidysheva, E.V.
Smirnova, T.A.
Tyukina

**Law of the Minimum Paradoxes**, e-print http://arxiv.org/abs/0907.1965

The "law of the minimum" states that growth is controlled by the
scarcest resource (limiting factor) (Justus von Liebig (1840)). This concept
was originally applied to plant or crop growth and quantitatively supported by
many experiments. Some generalizations based on more complicated
"dose-response" curves were proposed. Violations of this law in
natural and experimental ecosystems were also reported. We study models of
adaptation in ensembles of similar organisms under load of environmental
factors and prove that violation of the Liebig law follows from adaptation
effects. If the fitness of an organism in fixed environment satisfies the law
of the minimum then adaptation equalizes the pressure of essential factors and
therefore acts against the law. This is the the law
of the minimum paradox: if for a randomly chosen pair
"organism--environment" the law of the minimum typically holds, then,
in a well-adapted system, we have to expect violations of this law. For the
opposite interaction of factors (a synergistic system of factors which amplify
each other) adaptation leads from factor equivalence to limitations by a
smaller number of factors. For analysis of adaptation we develop a system of models
based on Selye's idea of the universal adaptation
resource (adaptation energy). These models predict that under the load of an
environmental factor a population separates into two groups (phases): a less
correlated, well adapted group and a highly correlated group with a larger
variance of attributes, which experiences problems with adaptation. Some
empirical data are presented and some evidences of interdisciplinary
applications to econometrics are discussed.

E. Chiavazzo, I. V. Karlin, A. N. Gorban and K Boulouchos,

**Combustion simulation via lattice Boltzmann
and reduced chemical kinetics,** J.
Stat. Mech. (2009) P06013, MIG-LB_StatMech_2009.pdf

We present and validate a methodology for coupling reduced models of detailed combustion mechanisms within the lattice Boltzmann framework. A detailed mechanism (9 species, 21 elementary reactions) for modeling reacting mixtures of air and hydrogen is considered and reduced using the method of invariant grids (MIG). In particular, a 2D quasi-equilibrium grid is constructed, further refined via the MIG method, stored in the form of tables and used to simulate a 1D flame propagating freely through a homogeneous premixed mixture. Comparisons between the detailed and reduced models show that the technique presented enables one to achieve a remarkable speedup in the computations with excellent accuracy.

A. N. Gorban, E. V. Smirnova, T. A. Tyukina,

**Correlations, Risk
and Crisis: from Physiology to Finance**, e-print: http://arxiv.org/abs/0905.0129.
Available at SSRN: http://ssrn.com/abstract=1397677.

We study the dynamics of correlation and variance in systems under the load of environmental factors. A universal effect in ensembles of similar systems under load of similar factors is described: in crisis, typically, even before obvious symptoms of crisis appear, correlation increases, and, at the same time, variance (and volatility) increases too. After the crisis achieves its bottom, it can develop into two directions: recovering (both correlations and variance decrease) or fatal catastrophe (correlations decrease, but variance not). This effect is supported by many experiments and observation of groups of humans, mice, trees, grassy plants, and on financial time series. A general approach to explanation of the effect through dynamics of adaptation is developed. Different organization of interaction between factors (Liebig's versus synergistic systems) lead to different adaptation dynamics. This gives an explanation to qualitatively different dynamics of correlation under different types of load.

A. N. Gorban, O. Radulescu, A. Y. Zinovyev,

**Limitation and Asymptotology of Chemical
Reaction Networks**,
e-print: http://arxiv.org/abs/0903.5072

The concept of the limiting step is extended to the asymptotology of multiscale reaction networks. Complete theory for linear networks with well separated reaction rate constants is developed. We present algorithms for explicit approximations of eigenvalues and eigenvectors of kinetic matrix. Accuracy of estimates is proven. Performance of the algorithms is demonstrated on simple examples. Application of algorithms to nonlinear systems is discussed.

** **

A. Gorban, I. Tyukin, E. Steur, H. Nijmeijer

**Positive Invariance Lemmas for Control
Problems with Convergence to Lyapunov-unstable Sets**, e-print http://arxiv.org/abs/0901.3577

We provide Lyapunov-like characterizations of positive invariance, boundedness and convergence of non-trivial solutions for a class of systems with unstable invariant sets. The systems of this class comprise of a stable part coupled with a one-dimensional unstable or critically stable subsystem. Examples of these systems appear in the problems of nonlinear output regulation, parameter estimation and adaptive control. We demonstrate that, for a large class of systems with unstable equilibria and solutions that might escape to infinity in finite time, it is always possible to determine simple criteria for positive invariance and boundedness of the system's nontrivial solutions. Conversely, it is possible to characterize domains of initial conditions that lead to solutions escaping from the origin. In contrast to other works addressing convergence issues in unstable systems, our results do not rely on the availability of input-output gains or contraction rates that are usually required for the stable compartment.

** **

**Principal Graphs and Manifolds****,** e-print: http://arxiv.org/abs/0809.0490

In many physical statistical, biological and other investigations it is desirable to approximate a system of points by objects of lower dimension and/or complexity. For this purpose, Karl Pearson invented principal component analysis in 1901 and found "lines and planes of closest fit to system of points". The famous k-means algorithm solves the approximation problem too, but by finite sets instead of lines and planes. This chapter gives a brief practical introduction into the methods of construction of general principal objects, i.e. objects embedded in the "middle" of the multidimensional data set. As a basis, the unifying framework of mean squared distance approximation of finite datasets is selected. Principal graphs and manifolds are constructed as generalisations of principal components and k-means principal points. For this purpose, the family of expectation/maximisation algorithms with nearest generalisations is presented. Construction of principal graphs with controlled complexity is based on the graph grammar approach.

** **

**Ovidiu Radulescu****, ****Alexander N Gorban****,
****Andrei
Zinovyev,**** and ****Alain Lilienbaum**

**Robust simplifications of multiscale biochemical networks, **BMC Systems Biology 2008, 2:86 doi:10.1186/1752-0509-2-86

*The most accessed paper in BMC Systems
Biology in November*__ __*2008*

*Background*

Cellular processes such as metabolism, decision making in development and differentiation, signalling, etc., can be modeled as large networks of biochemical reactions. In order to understand the functioning of these systems, there is a strong need for general model reduction techniques allowing to simplify models without loosing their main properties. In systems biology we also need to compare models or to couple them as parts of larger models. In these situations reduction to a common level of complexity is needed.

*Results*

We propose a systematic treatment of model reduction of multiscale biochemical networks. First, we consider linear kinetic models, which appear as "pseudo-monomolecular" subsystems of multiscale nonlinear reaction networks. For such linear models, we propose a reduction algorithm which is based on a generalized theory of the limiting step that we have developed in (Gorban and Radulescu 2008). Second, for non-linear systems we develop an algorithm based on dominant solutions of quasi-stationarity equations. For oscillating systems, quasi-stationarity and averaging are combined to eliminate time scales much faster and much slower than the period of the oscillations. In all cases, we obtain robust simplifications and also identify the critical parameters of the model. The methods are demonstrated for simple examples and for a more complex model of NFkB pathway.

*Conclusions*

Our approach allows critical parameter identification and produces hierarchies of models. Hierarchical modeling is important in "middle-out" approaches when there is need to zoom in and out several levels of complexity. Critical parameter identification is an important issue in systems biology with potential applications to biological control and therapeutics. Our approach also deals naturally with the presence of multiple time scales, which is a general property of systems biology models.

A.N. Gorban and O. Radulescu,

**Dynamic
and Static Limitation in Multiscale Reaction Networks****,** Revisited, *Advances in Chemical Engineering *34, 103-173. GorbanRadulescuAdvChemEng2008.pdf

The concept of the limiting step gives the limit simplification: the whole network behaves as a single step. This is the most popular approach for model simplification in chemical kinetics. However, in its elementary form this idea is applicable only to the simplest linear cycles in steady states. For simple cycles the nonstationary behavior is also limited by a single step, but not the same step that limits the stationary rate. In this chapter, we develop a general theory of static and dynamic limitation for all linear multiscale networks. Our main mathematical tools are auxiliary discrete dynamical systems on finite sets and specially developed algorithms of ‘‘cycles surgery’’ for reaction graphs. New estimates of eigenvectors for diagonally dominant matrices are used.

Multiscale ensembles of reaction networks with well-separated constants are introduced and typical properties of such systems are studied. For any given ordering of reaction rate constants the explicit approximation of steady state, relaxation spectrum and related eigenvectors (‘‘modes’’) is presented. In particular, we prove that for systems with well-separated constants eigenvalues are real (damped oscillations are improbable). For systems with modular structure, we propose the selection of such modules

that it is possible to solve the kinetic equation for every module in the explicit form. All such ‘‘solvable’’ networks are described. The obtained multiscale approximations, that we call ‘‘dominant systems’’ are computationally cheap and robust. These dominant systems can be used for direct computation of steady states and relaxation dynamics, especially when kinetic information is incomplete, for design of experiments and mining of experimental data, and could serve as a robust first approximation in perturbation theory or for preconditioning.

R. A. Brownlee, A. N.
Gorban, and J. Levesley,

**Nonequilibrium**** entropy limiters in lattice Boltzmann
methods**,** ****Physica**** A: Statistical Mechanics
and its Applications**

Volume 387, Issues 2-3, 15 January 2008, Pages 385-406 BrownGorbLevPhysA2007FinFin.pdf

We construct a system of nonequilibrium entropy
limiters for the lattice Boltzmann methods (LBM). These limiters erase spurious
oscillations without blurring of shocks, and do not affect smooth solutions. In
general, they do the same work for LBM as flux limiters do for finite
differences, finite volumes and finite elements methods, but for LBM the main
idea behind the construction of nonequilibrium
entropy limiter schemes is to transform a field of a scalar quantity — nonequilibrium entropy. There are two families of limiters:
(i) based on restriction of nonequilibrium
entropy (entropy “trimming”) and (ii) based on filtering of nonequilibrium
entropy (entropy filtering). The physical properties of LBM provide some
additional benefits: the control of entropy production and accurate estimation
of introduced artificial dissipation are possible. The constructed limiters are
tested on classical numerical examples: 1D athermal
shock tubes with an initial density ratio 1:2 and the 2D lid-driven cavity for
Reynolds numbers between 2000 and 7500 on a coarse 100×100
grid. All limiter constructions are applicable both for entropic and for
non-entropic equilibria.

**2007**

A. Gorban, B. Kegl, D. Wunsch, A. Zinovyev (Eds.),

**Principal Manifolds for Data Visualisation and
Dimension Reduction**, *Lecture Notes in
Computational Science and Engineering, Vol. 58*, Springer,
Berlin – Heidelberg – New York, 2007. (ISBN 978-3-540-73749-0)

In 1901, Karl Pearson invented Principal Component Analysis (PCA). Since then, PCA serves as a prototype for many other tools of data analysis, visualization and dimension reduction: Independent Component Analysis (ICA), Multidimensional Scaling (MDS), Nonlinear PCA (NLPCA), Self Organizing Maps (SOM), etc. The book starts with the quote of the classical Pearson definition of PCA and includes reviews of various methods: NLPCA, ICA, MDS, embedding and clustering algorithms, principal manifolds and SOM. New approaches to NLPCA, principal manifolds, branching principal components and topology preserving mappings are described as well. Presentation of algorithms is supplemented by case studies, from engineering to astronomy, but mostly of biological data: analysis of microarray and metabolite data. The volume ends with a tutorial "PCA and K-means decipher genome". The book is meant to be useful for practitioners in applied data analysis in life sciences, engineering, physics and chemistry; it will also be valuable to PhD students and researchers in computer sciences, applied mathematics and statistics.

A. N. Gorban,

**Selection Theorem for Systems with
Inheritance**, *Math.
Model. Nat. Phenom., *Vol. 2, No. 4, 2007, pp. 1-45. GOtborMMNP2(4)2007.pdf The original
publication is available at www.edpsciences.org

The
problem of finite-dimensional asymptotics of
infinite-dimensional dynamic systems is studied. A non-linear kinetic system
with *conservation of supports *for distributions has generically
finite-dimensional asymptotics. Such systems are
apparent in many areas of biology, physics (the theory of parametric wave
interaction), chemistry and economics. This conservation of support has a
biological interpretation: *inheritance*. The finite-dimensional asymptotics demonstrates effects of *natural selection*.
Estimations of the asymptotic dimension are presented. After some initial time,
solution of a kinetic equation with conservation of support becomes a finite
set of narrow peaks that become increasingly narrow over time and move
increasingly slowly. It is possible that these peaks do not tend to fixed
positions, and the path covered tends to infinity as *t→∞*. The *drift
equations *for peak motion are obtained. Various types of distribution
stability are studied: internal stability (stability with respect to
perturbations that do not extend the support), external stability or uninvadability (stability with respect to strongly small
perturbations that extend the support), and stable realizability
(stability with respect to small shifts and extensions of the density peaks).
Models of self-synchronization of cell division are studied, as an example of
selection in systems with additional symmetry. Appropriate construction of the
notion of typicalness in infinite-dimensional space
is discussed, and the notion of “completely thin” sets is introduced.

A.N. Gorban and O. Radulescu

**Dynamical robustness of biological
networks with hierarchical distribution of time scales**, IET
Syst. Biol.,
2007, 1, (4), pp. 238–246 Gorban2007IEESystemsBiology.pdf

Concepts of distributed robustness and r-robustness proposed by biologists to explain a variety of stability phenomena in molecular biology are analysed. Then, the robustness of the relaxation time using a chemical reaction description of genetic and signalling networks is discussed. First, the following result for linear networks is obtained: for large multiscale systems with hierarchical distribution of time scales, the variance of the inverse relaxation time (as well as the variance of the stationary rate) is much lower than the variance of the separate constants. Moreover, it can tend to 0 faster than 1/n, where n is the number of reactions. Similar phenomena are valid in the nonlinear case as well. As a numerical illustration, a model of signalling network is used for the important transcription factor NFkB.

A.N. Gorban and A.Y. Zinovyev

**The Mystery of Two
Straight Lines in Bacterial Genome Statistics****, **Bulletin of Mathematical
Biology
(2007) DOI 10.1007/s11538-007-9229-6 (Online First) GorbanZinovyev2007BMB1.pdf

In special coordinates (codon position-specific nucleotide frequencies),
bacterial genomes form two straight lines in 9-dimensional space: one line for
eubacterial genomes, another for archaeal genomes. All the 348 distinct bacterial
genomes available in Genbank in April 2007, belong to
these lines with high accuracy. The main challenge now is to explain the
observed high accuracy. The new phenomenon of complementary symmetry for codon
position-specific nucleotide frequencies is observed. The results of analysis
of several codon usage models are presented.We
demonstrate that the mean-field approximation, which is also known as
context-free, or complete independence model, or Segre variety, can serve as a
reasonable approximation to the real codon usage. The first two principal
components of codon usage correlate strongly with genomic G+C content and the
optimal growth temperature, respectively. The variation of codon usage along
the third component is related to the curvature of the mean-field
approximation. First three eigenvalues in codon usage PCA explain 59.1%, 7.8%
and 4.7% of variation. The eubacterial and archaeal genomes codon usage is
clearly distributed along two third order curves with genomic G+C content as a
parameter.

A.N.
Gorban, O.
Radulescu

**Dynamic and static limitation in reaction
networks, revisited,** http://arxiv.org/abs/physics/0703278 [physics.chem-ph] GorRadLimarXiv0703278v2.pdf

The concept of limiting step gives the limit simplification: the whole network
behaves as a single step. This is the most popular approach for model
simplification in chemical kinetics. However, in its simplest form this idea is
applicable only to the simplest linear cycles in steady states. For such the
simplest cycles the nonstationary behaviour is also
limited by a single step, but not the same step that limits the stationary
rate. In this paper, we develop a general theory of static and dynamic
limitation for all linear multiscale networks, not only for simple cycles. Our
main mathematical tools are auxiliary discrete dynamical systems on finite sets
and specially developed algorithms of ``cycles surgery" for reaction
graphs. New estimates of eigenvectors for diagonally dominant matrices are
used.

Multiscale ensembles of reaction networks with well separated constants are introduced and typical properties of such systems are studied. For any given ordering of reaction rate constants the explicit approximation of steady state, relaxation spectrum and related eigenvectors (``modes") is presented. In particular, we proved that for systems with well separated constants eigenvalues are real (damped oscillations are improbable). For systems with modular structure, we propose to select such modules that it is possible to solve the kinetic equation for every module in the explicit form. All such ``solvable" networks are described. The obtained multiscale approximations that we call ``dominant systems" are computationally cheap and robust. These dominant systems can be used for direct computation of steady states and relaxation dynamics, especially when kinetic information is incomplete, for design of experiments and mining of experimental data, and could serve as a robust first approximation in perturbation theory or for preconditioning.

R.A.
Brownlee, A.N. Gorban, J.
Levesley,

**Nonequilibrium**** entropy limiters in lattice Boltzmann
methods**, arXiv:0704.0043v1 [cond-mat.stat-mech] BrowGorLevLimitersArXiv.pdf

We construct a system of nonequilibrium entropy limiters for the lattice Boltzmann methods (LBM). These limiters erase spurious oscillations without blurring of shocks, and do not affect smooth solutions. In general, they do the same work for LBM as flux limiters do for finite differences, finite volumes and finite elements methods, but for LBM the main idea behind the construction of nonequilibrium entropy limiter schemes is to transform a field of a scalar quantity - nonequilibrium entropy. There are two families of limiters: (i) based on restriction of nonequilibrium entropy (entropy "trimming") and (ii) based on filtering of nonequilibrium entropy (entropy filtering). The physical properties of LBM provide some additional benefits: the control of entropy production and accurate estimate of introduced artificial dissipation are possible. The constructed limiters are tested on classical numerical examples: 1D athermal shock tubes with an initial density ratio 1:2 and the 2D lid-driven cavity for Reynolds numbers Re between 2000 and 7500 on a coarse 100*100 grid. All limiter constructions are applicable for both entropic and non-entropic quasiequilibria.

R. A. Brownlee, A. N.
Gorban, and J. Levesley,

**Stability and stabilization of the lattice
Boltzmann method****, **Phys. Rev. E **75**, 036711 (2007) *(17
pages)* BGJPhyRev2007.pdf

We^{ }revisit the classical stability versus accuracy dilemma for the
lattice^{ }Boltzmann methods (LBM). Our goal is a stable method of^{
}second-order accuracy for fluid dynamics based on the lattice Bhatnager-Gross-Krook^{ }method
(LBGK). The LBGK scheme can be recognized as a^{ }discrete dynamical
system generated by free flight and entropic involution.^{ }In this
framework the stability and accuracy analysis are more^{ }natural. We find
the necessary and sufficient conditions for second-order^{ }accurate
fluid dynamics modeling. In particular, it is proven
that^{ }in order to guarantee second-order accuracy the distribution
should belong^{ }to a distinguished surface—the invariant film (up to
second order^{ }in the time step). This surface is the trajectory of^{
}the (quasi)equilibrium distribution surface under free flight. The main
instability^{ }mechanisms are identified. The simplest recipes for
stabilization add no^{ }artificial dissipation (up to second order) and
provide second-order accuracy^{ }of the method. Two other prescriptions
add some artificial dissipation^{ }locally and prevent the system from
loss of positivity and^{ }local blowup.
Demonstration of the proposed stable LBGK schemes are^{ }provided by
the numerical simulation of a one-dimensional (1D) shock^{ }tube and
the unsteady 2D flow around a square cylinder^{ }up to Reynolds number
Re~20,000.

E. Chiavazzo, A.N. Gorban, and I.V. Karlin,

**Comparison
of Invariant Manifolds for Model Reduction in Chemical Kinetics****, **Commun. Comput.
Phys. Vol. **2**, No. 5 (2007), pp. 964-992 CiCP2007vol2_n5_p964.pdf

A modern approach to model reduction in chemical kinetics is often based on the notion of slow invariant manifold. The goal of this paper is to give a comparison of various methods of construction of slow invariant manifolds using a simple Michaelis-Menten catalytic reaction. We explore a recently introduced Method of Invariant Grids (MIG) for iteratively solving the invariance equation. Various initial approximations for the grid are considered such as Quasi Equilibrium Manifold, Spectral Quasi Equilibrium Manifold, Intrinsic Low Dimensional Manifold and Symmetric Entropic Intrinsic Low Dimensional Manifold. Slow invariant manifold was also computed using the Computational Singular Perturbation (CSP) method. A comparison between MIG and CSP is also reported.

A.N. Gorban,
N.R. Sumner, and A.Y. Zinovyev,

**Topological
grammars for data approximation**, Applied
Mathematics Letters Volume 20, Issue 4 (2007),
382-386 GorSummnZinAML2006.pdf

A method of *topological grammars* is proposed for
multidimensional data approximation. For data with complex topology we define a
*principal cubic complex* of low dimension and given complexity that gives
the best approximation for the dataset. This complex is a generalization of
linear and non-linear principal manifolds and includes them as particular
cases. The problem of optimal principal complex construction is transformed
into a series of minimization problems for quadratic functionals.
These quadratic functionals have a physically
transparent interpretation in terms of elastic energy. For the energy
computation, the whole complex is represented as a system of nodes and springs.
Topologically, the principal complex is a product of one-dimensional continuums
(represented by graphs), and the grammars describe how these continuums
transform during the process of optimal complex construction. This
factorization of the whole process onto one-dimensional transformations using
minimization of quadratic energy functionals allows
us to construct efficient algorithms.

A.N. Gorban,

**Order–disorder separation: Geometric
revision**, Physica A Volume 374, Issue 1 , 15 January 2007,
Pages 85-102 GorPhysA2006Order.pdf

After Boltzmann and Gibbs, the notion of disorder in statistical physics
relates to ensembles, not to individual states. This disorder is measured by
the logarithm of ensemble volume, the entropy. But recent results about measure
concentration effects in analysis and geometry allow us to return from the
ensemble-based point of view to a state-based one, at least, partially. In this
paper, the order–disorder problem is represented as a problem of relation
between distance and measure. The effect of strong order–disorder separation
for multiparticle systems is described: the phase
space could be divided into two subsets, one of them (set of disordered states)
has almost zero diameter, the second one has almost
zero measure. The symmetry with respect to permutations of particles is
responsible for this type of concentration. Dynamics of systems with strong
order–disorder separation has high average acceleration squared, which can be
interpreted as evolution through a series of collisions (acceleration-dominated
dynamics). The time arrow direction from order to disorder follows from the
strong order–disorder separation. But, inverse, for systems in space of
symmetric configurations with “sticky boundaries” the way back from disorder to
order is typical (Natural selection). Recommendations for mining of molecular
dynamics results are also presented.

**2006**** **

Ovidiu Radulescu, Alexander N. Gorban,
Sergei Vakulenko, Andrei Zinovyev

**Hierarchies
and modules in complex biological systems**, In: Proceedings of European
Conference on Complex Systems (paper ECCS06-114), Oxford, UK, September 2006 OxfordHiModP114.pdf

We review several
mathematical methods allowing to identify modules and hierarchies with several
levels of complexity in biological systems. These methods are based either on
the properties of the input-output characteristic of the modules or on global
properties of the dynamics such as the distribution of timescales or the
stratification of attractors with variable dimension. We also discuss the
consequences of the hierarchical structure on the robustness of biological processes.
Stratified attractors lead to Waddington's type canalization effects.
Successive application of the many to one mapping relating parameters of
different levels in an hierarchy of models (analogue
to the renormalization operation from statistical mechanics) leads to
concentration and robustness of those properties that are common to many levels
of complexity. Examples such as the response of the transcription factor NF*·*B
to signalling, and the segmentation patterns in the development of Drosophila
are used as illustrations of the theoretical ideas.

Gorban, A., Zinovyev, A., Popova,
T.

**Universal
Seven-Cluster Structure of Genome Fragment Distribution: Basic Symmetry in
Triplet Frequencies**, in Bioinformatics of Genome Regulation and
Structure, Kolchanov, Nikolay, Hofestaedt,
Ralf, Milanesi, Luciano (eds.), Springer US, 2006,
pp. 153-163.

We found a universal seven-cluster structure in bacterial genomic sequences and
explained its properties. Based on the analysis of 143 completely sequenced
bacterial genomes available in GenBank in August
2004, we show that there are four 'pure' types of the seven-cluster structure
observed. The type of cluster structure depends on GC content and reflects
basic symmetry in triplet frequencies. Animated 3D-scatters of
bacterial genomes seven-cluster structure are available on our web site: http://www.ihes.fr/~zinovyev/7clusters
.

R. A. Brownlee, A. N. Gorban, and J. Levesley,

**Stabilization
of the lattice Boltzmann method using the Ehrenfests'
coarse-graining idea,** Phys. Rev. E **74**, 037703 (2006) RobBrowGorbLeveslPRE2006.pdf

The^{ }lattice Boltzmann method (LBM) and its variants have emerged as^{
}promising, computationally efficient and increasingly popular numerical
methods for modeling^{ }complex fluid flow.
However, it is acknowledged that the method^{ }can demonstrate
numerical instabilities, e.g., in the vicinity of shocks.^{ }We propose
a simple technique to stabilize the LBM by^{ }monitoring the difference
between microscopic and macroscopic entropy. Populations are^{ }returned
to their equilibrium states if a threshold value is^{ }exceeded. We coin
the name *Ehrenfests**' steps* for this
procedure^{ }in homage to the vehicle that we use to introduce^{ }the
procedure, namely, the Ehrenfests' coarse-graining
idea.

A.N. Gorban, B.M. Kaganovich,
S.P. Filippov, A.V. Keiko, V.A. Shamansky,
I.A. Shirkalin,

**Thermodynamic
Equilibria and Extrema: Analysis of Attainability Regions and Partial
Equilibria**, Springer, Berlin-Heidelberg-New York, 2006.

**Model
Reduction and Coarse--Graining Approaches for Multiscale Phenomena,
**Ed. by Alexander N. Gorban, Nikolaos Kazantzis, Ioannis G. Kevrekidis, Hans
Christian Öttinger, Constantinos
Theodoropoulos ,

A. Gorban, I. Karlin, A. Zinovyev,

**Invariant Grids: Method of Complexity
Reduction in Reaction Networks,** Complexus, V. 2, 110–127. ComPlexUs2006.pdf

Complexity in the description of big chemical reaction networks has both structural (number of species and reactions) and temporal (very different reaction rates) aspects. A consistent way to make model reduction is to construct the invariant manifold which describes the asymptotic system behaviour. In this paper we present a discrete analogue of this object: an invariant grid. The invariant grid is introduced independently from the invariant manifold notion and can serve to represent the dynamic system behaviour as well as to approximate the invariant manifold after refinement. The method is designed for pure dissipative systems and widely uses their thermodynamic properties but allows also generalizations for some classes of open systems. The method is illustrated by two examples: the simplest catalytic reaction (Michaelis-Menten mechanism) and the hydrogen oxidation.

A.N. Gorban,

**Basic Types of Coarse-Graining, **e-print http://arxiv.org/abs/cond-mat/0602024
(local copy CoaGrWorkSpri7.pdf).

42 pgs, 11 figs. A talk given at the research workshop: "Model Reduction and Coarse-Graining Approaches for Multiscale Phenomena,"

We consider two basic types of coarse-graining: the Ehrenfest's coarse-graining and its extension to a general principle of non-equilibrium thermodynamics, and the coarse-graining based on uncertainty of dynamical models and $\epsilon$-motions (orbits). Non-technical discussion of basic notions and main coarse-graining theorems are presented: the theorem about entropy overproduction for the Ehrenfest's coarse-graining and its generalizations, both for conservative and for dissipative systems, and the theorems about stable properties and the Smale order for $\epsilon$-motions of general dynamical systems including structurally unstable systems. A brief discussion of two other types, coarse-graining by rounding and by small noise, is also presented. Computational kinetic models of macroscopic dynamics are considered. We construct a theoretical basis for these kinetic models using generalizations of the Ehrenfest's coarse-graining.

A.N. Gorban, I.V. Karlin,**
Quasi-Equilibrium Closure Hierarchies for the
Boltzmann Equation**, Physica A 360 (2006)
325–364 GKQEBoltzPhysA2006.pdf

In this paper, explicit method of constructing approximations (the Triangle Entropy Method) is developed for nonequilibrium problems. This method enables one to treat any complicated nonlinear functionals that fit best the physics of a problem (such as, for example, rates of processes) as new independent variables.

The work of the method was demonstrated on the Boltzmann's - type kinetics. New macroscopic variables are introduced (moments of the Boltzmann collision integral, or scattering rates). They are treated as independent variables rather than as infinite moment series. This approach gives the complete account of rates of scattering processes. Transport equations for scattering rates are obtained (the second hydrodynamic chain), similar to the usual moment chain (the first hydrodynamic chain). Various examples of the closure of the first, of the second, and of the mixed hydrodynamic chains are considered for the hard spheres model. It is shown, in particular, that the complete account of scattering processes leads to a renormalization of transport coefficients.

The method gives the explicit solution for the closure problem, provides thermodynamic properties of reduced models, and can be applied to any kinetic equation with a thermodynamic Lyapunov function

A. Gorban, A. Zinovyev,

**Elastic
Principal Graphs and Manifolds and their Practical Applications,**
Computing 75, 359–379 (2005), (DOI) **10.1007/s00607-005-0122-6**
, GorbZin2005Computing.pdf

Principal manifolds serve as useful tool for many practical
applications. These manifolds are defined as lines or surfaces passing through
“the middle” of data distribution. We propose an algorithm for fast
construction of grid approximations of principal manifolds with given topology.
It is based on analogy of principal manifold and elastic membrane. First
advantage of this method is a form of the functional to be minimized which
becomes quadratic at the step of the vertices position refinement. This makes
the algorithm very effective, especially for parallel implementations. Another
advantage is that the same algorithmic kernel is applied to construct principal
manifolds of different dimensions and topologies. We demonstrate how
flexibility of the approach allows numerous adaptive strategies like principal
graph constructing, etc. The algorithm is implemented as a C++ package *elmap** *and as a part of stand-alone data
visualization tool *VidaExpert*, available on
the web. We describe the approach and provide several examples of its
application with speed performance characteristics.

A.N. Gorban, I.V. Karlin,

**Invariance correction to Grad's equations: Where to go beyond
approximations?** Continuum Mechanics and Thermodynamics, 17(4) (2005), 311–335, GorKarCMT_05.pdf, http://arxiv.org/abs/cond-mat/0504221

We review some recent developments of Grad's approach
to solving the Boltzmann equation and creating reduced description. The method
of invariant manifold is put forward as a unified principle to establish
corrections to Grad's equations. A consistent derivation of regularized Grad's
equations in the framework the method of invariant manifold is given. A new
class of kinetic models to lift the finite-moment description to a kinetic
theory in the whole space is established. Relations of Grad's approach to
modern mesoscopic integrators such as the entropic lattice Boltzmann method are
also discussed.

A.N.
Gorban, T.G.Popova, A.Yu. Zinovyev,

**Codon usage trajectories and 7-cluster
structure of 143 complete bacterial genomic sequences ***Physica A
*353C (2005), 365-387. CodonPhysA2005.pdf (Number 11 in **TOP25
articles within the journal:** Physica A: Statistical Mechanics and its
Applications, APR - JUN 2005 Top25.pdf)

Three results are presented. First, we prove the existence of a universal
7-cluster structure in all 143 completely sequenced bacterial genomes available
in Genbank in August 2004, and explained its
properties. The 7-cluster structure is responsible for the main part of
sequence heterogeneity in bacterial genomes. In this sense, our 7 clusters is
the basic model of bacterial genome sequence. We demonstrated that there are
four basic ``pure" types of this model, observed in nature: ``parallel
triangles", ``perpendicular triangles", degenerated case and the
flower-like type.

Second, we answered the question: how big are the position-specific information
and the contribution connected with correlations between nucleotide. The
accuracy of the mean-field (context-free) approximation is estimated for
bacterial genomes.

We show that codon usage of bacterial genomes is a multi-linear function of
their genomic G+C-content with high accuracy (more precisely, by two similar
functions, one for eubacterial genomes and the other one for archaea).
Description of these two codon-usage trajectories is the third result.

All 143 cluster animated 3D-scatters are collected in a database and is made
available on our web-site: http://www.ihes.fr/~zinovyev/7clusters
.

A.N. Gorban, T.G.Popova,
A.Yu. Zinovyev,**
Four basic symmetry types in the universal 7-cluster structure of microbial
genomic sequences,** In
Silico Biology, 5 (2005), 0039. Internet site CLUSTER STRUCTURE IN
GENOME with analysis of all bacterial genomes.

Coding information is the main source of heterogeneity (non-randomness) in the sequences of microbial genomes. The heterogeneity corresponds to a cluster structure in triplet distributions of relatively short genomic fragments (200-400bp). We found a universal 7-cluster structure in microbial genomic sequences and explained its properties. We show that codon usage of bacterial genomes is a multi-linear function of their genomic G+C-content with high accuracy. Based on the analysis of 143 completely sequenced bacterial genomes available in Genbank in August 2004, we show that there are four "pure" types of the 7-cluster structure observed. All 143 cluster animated 3D-scatters are collected in a database which is made available on our web-site (http://www.ihes.fr/~zinovyev/7clusters). The findings can be readily introduced into software for gene prediction, sequence alignment or microbial genomes classification.

A.N. Gorban, I.V. Karlin,

**Invariant Manifolds for Physical and
Chemical Kinetics,** Lect.
Notes Phys. 660, Springer, Berlin, Heidelberg, 2005 (498 pages). [Preface-Contents-Introduction(pdf)]
[Review
in *Bull. London Math. Soc**. *38 (2006) (pdf)] [Review
in Zentralblatt Math. (2006) (pdf)]
[Editorial Reviews(htm)] *Russian
web-site with this book*

The concept of the slow invariant manifold is recognized as the central idea
underpinning a transition from micro to macro and model reduction in kinetic
theories. We present the constructive methods of invariant manifolds for model
reduction in physical and chemical kinetics, developed during last two decades.
The physical problem of reduced description is studied in the most general form
as a problem of constructing the slow invariant manifold. The invariance
conditions are formulated as the differential equation for a manifold immersed
in the phase space (** the invariance equation**). The equation of motion for immersed
manifolds is obtained (

A collection of methods to derive analytically and to compute numerically the slow invariant manifolds is presented. Among them, iteration methods based on incomplete linearization, relaxation method and the method of invariant grids are developed. The systematic use of thermodynamic structures and of the quasi-chemical representation allows us to construct approximations which are in concordance with physical restrictions.

The following examples of applications are presented: Nonperturbative derivation of physically consistent hydrodynamics from the Boltzmann equation and from the reversible dynamics, for Knudsen numbers Kn~1; construction of the moment equations for nonequilibrium media and their dynamical correction (instead of extension of the list of variables) in order to gain more accuracy in description of highly nonequilibrium flows; kinetic theory of phonons; model reduction in chemical kinetics; derivation and numerical implementation of constitutive equations for polymeric fluids; the limits of macroscopic description for polymer molecules, cell division kinetics.

**Keywords:** Model Reduction;
Invariant Manifold; Entropy; Kinetics; Boltzmann Equation; Fokker--Planck
Equation; Navier-Stokes Equation; Burnett Equation;
Quasi-chemical Approximation; Oldroyd Equation;
Polymer Dynamics; Molecular Individualism; Accuracy Estimation;
Post-processing.

**PACS codes:** 05.20.Dd Kinetic
theory, 02.30.Mv Approximations and expansions, 02.70.Dh Finite-element and Galerkin methods, 05.70.Ln Nonequilibrium
and irreversible thermodynamics.

**A.N. Gorban
**

After Boltzmann and Gibbs, the notion of disorder in statistical physics relates to ensembles, not to individual states. This disorder is measured by the logarithm of ensemble volume, the entropy. But recent results about measure concentration effects in analysis and geometry allow us to return from the ensemble--based point of view to a state--based one, at least, partially. In this paper, the order--disorder problem is represented as a problem of relation between distance and measure. The effect of strong order--disorder separation for multiparticle systems is described: the phase space could be divided into two subsets, one of them (set of disordered states) has almost zero diameter, the second one has almost zero measure. The symmetry with respect to permutations of particles is responsible for this type of concentration. Dynamics of systems with strong order--disorder separation has high average acceleration squared, which can be interpreted as evolution through a series of collisions (acceleration--dominated dynamics). The time arrow direction from order to disorder follows from the strong order--disorder separation. But, inverse, for systems in space of symmetric configurations with ``sticky boundaries" the way back from disorder to order is typical (Natural selection). Recommendations for mining of molecular dynamics results are presented also.

**S. Ansumali, S. Archidiacono,
S. Chikatamarla, A.N. Gorban,
I.V. Karlin,
**

A new approach to model hydrodynamics at the level of one-particle distribution function is presented. The construction is based on the choice of quasi-equilibria pertinent to the physical context of the problem. Kinetic equations for a single component fluid with a given Prandtl number and models of mixtures with a given Schmidt number are derived. A novel realization of these models via an auxiliary kinetic equation is suggested.

**A.N.
Gorban, G.S. Yablonsky
**

Everything that is not prohibited is permissible. So, what is prohibited in the course of chemical reactions, heat transfer and other dissipative processes? Is it possible to "overshoot" the equilibrium, and if yes, then how far? Thermodynamically allowed and prohibited trajectories of processes are discussed by the example of effects of equilibrium encircling. The complete theory of thermodynamically accessible states is presented. The space of all thermodynamically admissible paths is presented by projection on the "thermodynamic tree", that is the tree of the related thermodynamic potential (entropy, free energy, free enthalpy) in the balance polyhedron. The stationary states and limit points for open systems are localized too.

**A.N.
Gorban, M. Kudryashev, T. Popova,
**

What proteins are made from, as the working parts of the living cells protein machines? To answer this question, we need a technology to disassemble proteins onto elementary functional details and to prepare lumped description of such details. This lumped description might have a multiple material realization (in amino acids). Our hypothesis is that informational approach to this problem is possible. We propose a way of hierarchical classification that makes the primary structure of protein maximally non-random and compare them with other classifications. The first step of the suggested research program is realized: the analysis of protein binary alphabet in comparison with other amino acid classifications.

**A.N. Gorban,
A. Yu. Zinovyev
**

In this paper, we give a tutorial for undergraduate students studying statistical methods and/or bioinformatics. The students learn how data visualization can help in genomic sequences analysis. Students start with a fragment of genetic text of a bacterial genome and analyze its structure. By means of principal component analysis they ``discover'' that the information in genome is encoded by non-overlapping triplets. Next, they learn to find gene positions. This exercise on principal component analysis and K-Means clustering gives a possibility for active study of the basic bioinformatics notions. In Appendix the program listings for MatLab are published.

**2004**

**A.N. Gorban,
D.A. Rossiyev, M.G. Dorrer
**

This work describes neural software applied in medicine and physiology to: investigate and diagnose immune deficiencies; diagnose and study allergic and pseudoallergic reactions; forecast emergence or aggravation of stagnant cardiac insufficiency in patients with cardiac rhythm disorders; forecast development of cardiac arrhythmia after myocardial infarction; reveal relationships between the accumulated radiation dose and a set of immunological, hormonal, and bio-chemical parameters of human blood and find a method to be able to judge by these parameters the dose value; propose a technique for early diagnosis of chor-oid melanomas; Neural networks also help to predict human relations within a group.

**A.N.
Gorban, A.Yu. Zinovyev,
**

In special coordinates (codon position--specific nucleotide frequencies) bacterial genomes form two straight lines in 9-dimensional space: one line for eubacterial genomes, another for archaeal genomes. All the 175 known bacterial genomes (Genbank, March 2005) belong these lines with high accuracy, and these two lines are certainly different. The results of PCA analysis of codon usage and accuracy of mean--field (context--free) approximation are presented. The first two principal components correlate strongly with genomic G+C-content and the optimal growth temperature respectively. The variation of codon usage along the third component is related to the curvature of the mean-field approximation. The eubacterial and archaeal genomes codon usage are clearly distributed along two third order curves with genomic G+C-content as a parameter.

A.N. Gorban, T.G. Popova, A.Yu.
Zinovyev,

**Four basic symmetry types in the universal 7-cluster structure of 143
complete bacterial genomic sequences** E-print: http://arxiv.org/abs/q-bio/0410033

The coding information is the main source of heterogeneity (non-randomness) in
the sequences of bacterial genomes. This information can be naturally modeled by analysing cluster structures in the
"in-phase" triplet distributions of relatively short genomic
fragments (200-400bp). We found a universal 7-cluster structure in bacterial
genomic sequences and explained its properties. We show that codon usage of
bacterial genomes is a multi-linear function of their genomic G+C-content with
high accuracy. Based on the analysis of 143 completely sequenced bacterial genomes
available in Genbank in August 2004, we show that
there are four "pure" types of the 7-cluster structure observed. All
143 cluster animated 3D-scatters are collected in a database and is made
available on our web-site: http://www.ihes.fr/~zinovyev/7clusters.
The finding can be readily introduced into any software for gene prediction,
sequence alignment or bacterial genomes classification.

Gorban A.N., Popova T.G., Zinovyev A.Yu.,

**Seven clusters
and unsupervised gene prediction,** Proceedings of the Fourth
International Conference on Bioinformatics of Genome Regulation and Structure, BGRS’ 2004, Novosibirsk, Russia, July 25 -
30, 2004, IC&G, Novosibirsk, 2004, pp. 277-280.

*Motivation:
*The effectiveness of most unsupervised gene-detection algorithms follows
from a cluster structure in oligomer distributions. Existence of this structure
is implicitly known but it was never visualized and studied in terms of data
exploration strategies. Visual representation of the structure allows deeper
understanding of its properties and can serve to display and analyze characteristics of existing gene-finders.

*Results:
*The cluster structure of genome fragments distribution in the space of
their triplet frequencies was revealed by pure data exploration strategy.
Several complete genomic sequences were analyzed,
using visualization of distribution of 64-dimensional vectors of triplet
frequencies in a sliding window. The structure of distribution was found to
consist of seven clusters, corresponding to proteincoding
genome fragments in three possible phases in each of the two complementary
strands and to the non-coding regions with high accuracy. The self-training
technique for automated gene recognition both in entire genomes and in
unassembled ones is proposed.

Gorban, A.N., Zinovyev, A.Y.

**Elastic principal manifolds and their practical applications **E-print http://arxiv.org/abs/cond-mat/0405648

Principal manifolds defined as lines or surfaces passing through "the
middle" of the data distribution serve as useful objects for many
practical applications. We propose a new algorithm for fast construction of
grid approximations of principal manifolds with given topology. One advantage
of the method is a new form of the functional to be minimized, which becomes
quadratic at the step of the vertexes positions refinement. This makes the
algorithm very effective, especially for parallel implementations. Another
advantage is that the same algorithmic kernel is applied to construct principal
manifolds of different dimensions and topologies. We demonstrate how
flexibility of the approach allows easily numerous adaptive strategies like
principal graph constructing, etc. The algorithm is implemented as a C++
package elmap and as a part of stand-alone data
visualization tool VidaExpert, available on the web.
We describe the approach and provide several examples of its applications with
speed performance characteristics.

Gorban, A.N.

**Systems with inheritance: dynamics of
distributions with conservation of support, natural selection and
finite-dimensional asymptotics** E-print: http://arxiv.org/abs/cond-mat/0405451

If we find a representation of an infinite-dimensional dynamical system as a
nonlinear kinetic system with {\it conservation of supports} of distributions,
then (after some additional technical steps) we can state that the asymptotics is finite-dimensional. This conservation of
support has a {\it quasi-biological interpretation, inheritance} (if a gene was
not presented initially in a
isolated population without mutations, then it cannot appear at later time). These
quasi-biological models can describe various physical, chemical, and, of
course, biological systems. The finite-dimensional asymptotic demonstrates
effects of {\it "natural" selection}. The estimations of asymptotic
dimension are presented. The support of an individual limit distribution is
almost always small. But the union of such supports can be the whole space even
for one solution. Possible are such situations: a solution is a finite set of
narrow peaks getting in time more and more narrow,
moving slower and slower. It is possible that these peaks do not tend to fixed
positions, rather they continue moving, and the path covered tends to infinity
at $t \to \infty$. The {\it drift equations} for
peaks motion are obtained. Various types of stability are studied. In example,
models of cell division self-synchronization are studied. The appropriate
construction of notion of typicalness in
infinite-dimensional spaces is discussed, and the "completely thin"
sets are introduced

Gorban, A.N.

**Singularities of transition processes in
dynamical systems: Qualitative theory of critical delays ***Electron.
J. Diff. Eqns.* Monograph 5, 2004, 55 p.Slorelax2004EJDE.pdf
Online: http://ejde.math.txstate.edu/Monographs/05/abstr.html

This monograph presents a systematic analysis of the
singularities in the transition processes for dynamical systems. We study general
dynamical systems, with dependence on a parameter, and construct relaxation
times that depend on three variables: Initial conditions x, parameters k of the
system, and accuracy e of the relaxation. We study the singularities of
relaxation times as functions of (x,k)
under fixed e, and then classify the bifurcations (explosions) of limit sets.
We study the relationship between singularities of relaxation times and
bifurcations of limit sets. An analogue of the Smale
order for general dynamical systems under perturbations is constructed. It is
shown that the perturbations simplify the situation: the interrelations between
the singularities of relaxation times and other peculiarities of dynamics for
general dynamical system under small perturbations are the same as for the
Morse-Smale systems

Gorban, A.N.;Gorban, P.A.;Karlin, I.V.

**Legendre integrators, post-processing and
quasiequilibrium ***J. Non-Newtonian
Fluid Mech.* 120 (2004) 149-167 GoGoKar2004.pdf
Online: http://arxiv.org/abs/cond-mat/0308488

A toolbox for the development and reduction of the
dynamical models of nonequilibrium systems is
presented. The main components of this toolbox are: Legendre integrators,
dynamical post-processing, and the thermodynamic projector. The thermodynamic
projector is the tool to transform almost any anzatz
to a thermodynamically consistent model. The post-processing is the cheapestway to improve the solution obtained by the
Legendre integrators. Legendre integrators give the opportunity to solve linear
equations instead of nonlinear ones for quasiequilibrium
(maximum entropy, MaxEnt) approximations. The
essentially new element of this toolbox, the method of thermodynamic projector,
is demonstrated on application to the FENE-P model of polymer kinetic theory.
The multi-peak model of polymer dynamics is developed.

Gorban, A.N.;Karlin, I.V.

**Uniqueness of thermodynamic projector and
kinetic basis of molecular individualism** *Physica** A,* 336, 2004, 391-432 UniMolIndRepr.pdf Online: http://arxiv.org/abs/cond-mat/0309638

Three results are presented: First, we solve the problem of persistence of
dissipation for reduction of kinetic models. Kinetic equations with
thermodynamic Lyapunov functions are studied.
Uniqueness of the thermodynamic projector is proven: There exists only one
projector which transforms any vector field equipped with the given Lyapunov function into a vector field with the same Lyapunov function for a given anzatz
manifold which is not tangent to the Lyapunov
function levels. Second, we use the thermodynamic projector for developing the
short memory approximation and coarse-graining for general nonlinear dynamic
systems. We prove that in this approximation the entropy production increases.
(The theorem about entropy overproduction.) In example, we apply the
thermodynamic projector to derive the equations of reduced kinetics for the
Fokker-Planck equation. A new class of closures is developed, the kinetic multipeak polyhedra.
Distributions of this type are expected in kinetic models with multidimensional
instability as universally as the Gaussian distribution appears for stable
systems. The number of possible relatively stable states of a nonequilibrium system grows as 2^m, and the number of
macroscopic parameters is in order mn, where n is the
dimension of configuration space, and m is the number of independent unstable
directions in this space. The elaborated class of closures and equations
pretends to describe the effects of molecular individualism. This is the third
result.

Gorban, A.N.;Karlin, I.V.;Zinovyev, A.Y.

**Constructive methods of invariant manifolds for
kinetic problems** *Phys. Rep*.,
396, 2004, 197-403 PhysRepCorr.pdf Online: http://arxiv.org/abs/cond-mat/0311017

The concept of the slow invariant manifold is
recognized as the central idea underpinning a transition from micro to macro
and model reduction in kinetic theories. We present the Constructive Methods of
Invariant Manifolds for model reduction in physical and chemical kinetics,
developed during last two decades. The physical problem of reduced description
is studied in the most general form as a problem of constructing the slow
invariant manifold. The invariance conditions are formulated as the differential
equation for a manifold immersed in the phase space (the invariance equation).
The equation of motion for immersed manifolds is obtained (the film extension
of the dynamics). Invariant manifolds are fixed points for this equation, and
slow invariant manifolds are Lyapunov stable fixed
points, thus slowness is presented as stability.

A collection of methods to derive analytically and to compute numerically the
slow invariant manifolds is presented. Among them, iteration methods based on
incomplete linearization, relaxation method and the method of invariant grids
are developed. The systematic use of thermodynamics structures and of the
quasi-chemical representation allow to construct approximations which are in
concordance with physical restrictions.

The following examples of applications are presented: nonperturbative
derivation of physically consistent hydrodynamics from the Boltzmann equation
and from the reversible dynamics, for Knudsen numbers Kn~1; construction of the
moment equations for nonequilibrium media and their
dynamical correction (instead of extension of list of variables) to gain more
accuracy in description of highly nonequilibrium
flows; determination of molecules dimension (as diameters of equivalent hard
spheres) from experimental viscosity data ; model reduction in chemical
kinetics; derivation and numerical implementation of constitutive equations for
polymeric fluids; the limits of macroscopic description for polymer molecules,
etc.

Gorban, A.N.;Karlin, I.V.;Zinovyev, A.Y.

**Invariant grids for reaction kinetics ***Physica* A, 333,
2004 106-154 ChemGrPhA2004.pdf Online: http://arxiv.org/abs/cond-mat/0307076

In this paper, we review the construction of
low-dimensional manifolds of reduced description for equations of chemical
kinetics from the standpoint of the method of invariant manifold (MIM). MIM is
based on a formulation of the condition of invariance as an equation, and its
solution by Newton iterations. A grid-based version of MIM is developed.
Generalizations to open systems are suggested. The set of methods covered makes
it possible to effectively reduce description in chemical kinetics. The most
essential new element of this paper is the systematic consideration of a
discrete analogue of the slow (stable) positively invariant manifolds for
dissipative systems, {invariant grids}. We describe the Newton method and the
relaxation method for the invariant grids construction. The problem of the grid
correction is fully decomposed into the problems of the grid's nodes
correction. The edges between the nodes appears only in the calculation of the
tangent spaces. This fact determines high computational efficiency of the
invariant grids method.

A. Yu. Zinovyev, A. N. Gorban, T. G. Popova

Self-Organizing Approach for Automated Gene
Identification*Open Sys. &
Information Dyn.,* 10, 2003, 321-333 GoZiPo2003final.pdf

Self-training technique for automated gene recognition both in entire genomes
and in unassembled ones is proposed. It is based on a simple measure (namely,
the vector of frequencies of non-overlapping triplets in sliding window), and
needs neither predetermined information, nor preliminary learning. The sliding
window length is the only one tuning parameter. It should be chosen close to
the average exon length typical to the DNA text under investigation. An
essential feature of the technique proposed is preliminary visualization of the
set of vectors in the subspace of the first three principal components. It was
shown, the distribution of DNA sites has the bullet-like structure with one
central cluster (corresponding to non-coding sites) and three or six ank ones (corresponding to protein-coding sites). The
bullet-like structure itself revealed in the distribution seems to be very
interesting illustration of triplet usage in DNA sequence. The method was
examined on several genomes (mitochondrion of P.wickerhamii,
bacteria C.crescentus and primitive eukaryot S.cerevisiae). The
percentage of truly predicted nucleotides exceeds 90%.

*In October 2004 this paper was mentioned as one of the five most viewed
paper published in the Journal since 1997 *http://www.kluweronline.com/issn/1230-1612
.

A. N. Gorban, A. Yu. Zinovyev, T. G. Popova

**Seven clusters in genomic triplet distributions **In Silico Biology, 3, 2003, 471-482 (0039), Online: http://arXiv.org/abs/cond-mat/0305681
29 May 2003 Seven03.pdf

Motivation: In several recent papers new algorithms were proposed for detecting
coding regions without requiring learning dataset of already known genes. In
this paper we studied cluster structure of several genomes in the space of
codon usage. This allowed to interpret some of the results obtained in other studies
and propose a simpler method, which is, nevertheless, fully functional.
Results: Several complete genomic sequences were analyzed,
using visualization of tables of triplet counts in a sliding window. The
distribution of 64-dimensional vectors of triplet frequencies displays a
well-detectable cluster structure. The structure was found to consist of seven
clusters, corresponding to protein-coding information in three possible phases
in one of the two complementary strands and in the non-coding regions.
Awareness of the existence of this structure allows development of methods for
the segmentation of sequences into regions with the same coding phase and
non-coding regions. This method may be completely unsupervised or use some
external information. Since the method does not need extraction of ORFs, it can
be applied even for unassembled genomes. Accuracy calculated on the base-pair
level (both sensitivity and specificity) exceeds 90%. This is not worse as
compared to such methods as HMM, however, has the advantage to be much simpler
and clear. Availability: The software and datasets are available at http://www.ihes.fr/~zinovyev/bullet

Gorban, A.N.;Karlin,
I.V.,**
Method of invariant manifold for chemical kinetics**,

NEW:

In this paper, we review the construction of low-dimensional manifolds of reduced description for equations of chemical kinetics from the standpoint of the method of invariant manifold (MIM). The MIM is based on a formulation of the condition of invariance as an equation, and its solution by Newton iterations. A review of existing alternative methods is extended by a thermodynamically consistent version of the method of intrinsic low-dimensional manifolds. A grid-based version of the MIM is developed, and model extensions of low-dimensional dynamics are described. Generalizations to open systems are suggested. The set of methods covered makes it possible to effectively reduce description in chemical kinetics

A. N. Gorban, A. Y. Zinovyev, D.C. Wunsch

**Application of
The Method of Elastic Maps In Analysis of Genetic Texts,**** **Proceedings
of IJCNN2003 GZW2003.pdf

Method of elastic maps allows to construct efficiently 1D, 2D and 3D non-linear
approximations to the principal manifolds with different topology (piece of
plane, sphere, torus etc.) and to project data onto it. We describe the idea of
the method and demonstrate its applications in analysis of genetic
sequences.

Gorban A. N., Karlin I. V.

**Quasi-Equilibrium
Closure Hierarchies for The Boltzmann**** Equation **E-print, http://arXiv.org/abs/cond-mat/0305599
v1 26 May 2003 Triangl2003.pdf

Explicit method of constructing of approximations (Triangle Entropy Method) is
developed for strongly nonequilibrium problems of
Boltzmann's--type kinetics, i.e. when standard moment variables are
insufficient. This method enables one to treat any complicated nonlinear functionals that fit the physics of a problem (such as, for
example, rates of processes) as new independent variables. The method is
applied to the problem of derivation of hydrodynamics from the Boltzmann
equation. New macroscopic variables are introduced (moments of the Boltzmann
collision integral, or collision moments). They are treated as independent
variables rather than as infinite moment series. This approach gives the
complete account of rates of scattering processes. Transport equations for
scattering rates are obtained (the second hydrodynamic chain), similar to the
usual moment chain (the first hydrodynamic chain). Using the triangle entropy
method, three different types of the macroscopic description are considered. The
first type involves only moments of distribution functions, and results
coincide with those of the Grad method in the Maximum Entropy version. The
second type of description involves only collision moments. Finally, the third
type involves both the moments and the collision moments (the mixed
description). The second and the mixed hydrodynamics are sensitive to the
choice of the collision model. The second hydrodynamics is equivalent to the
first hydrodynamics only for Maxwell molecules, and the mixed hydrodynamics
exists for all types of collision models excluding Maxwell molecules. Various
examples of the closure of the first, of the second, and of the mixed
hydrodynamic chains are considered for the hard spheres model. It is shown, in
particular, that the complete account of scattering processes leads to a
renormalization of transport coefficients.

The paper gives English translation of the first part of the paper: Gorban, A.
N., Karlin, I. V., Quasi-equilibrium approximation and non-standard expansions in
the theory of the Boltzmann kinetic equation, in: "Mathematical Modelling
in Biology and Chemistry. New Approaches", ed. R. G. Khlebopros,
Nauka, Novosibirsk, P.69-117 (1992) [in Russian].

Gorban A. N.

**Neuroinformatics: What are us, where are we going, how to
measure our way?**** **The lecture was given at the USA-NIS Neurocomputing opportunities workshop, Washington DC, July
1999 (Associated with IJCNN'99) E-print: http://arxiv.org/abs/cond-mat/0307346

What is neuroinformatics? We
can define it as a direction of science and information technology, dealing
with development and study of the methods for solution of problems by means of
neural networks. A field of science cannot be determined only by fixing what it
is "dealing with". The main component, actually constituting a
scientific direction, is "THE GREAT PROBLEM", around which the
efforts are concentrated. One may state even categorically: if there is no a
great problem, there is no a field of science, but only more or less skilful
imitation. What is "THE GREAT PROBLEM" for neuroinformatics?
The problem of effective parallelism, the study of brain (solution of mysteries
of thinking), etc are discussed. The neuroinformatics was considered not only as a science, but
as a services sector too. The main ideas of generalized technology of
extraction of explicit knowledge from data are presented. The mathematical
achievements generated by neuroinformatics, the
problem of provability of neurocomputations, and
benefits of neural network realization of solution of a problem are discussed.

Gorban A. N., Karlin I. V.

**Geometry of irreversibility: The film of nonequilibrium****
states** E-print: http://arxiv.org/abs/cond-mat/0308331

A general geometrical framework of nonequilibrium
thermodynamics is developed. The notion of macroscopically definable ensembles
is developed. The thesis about macroscopically definable ensembles is
suggested. This thesis should play the same role in the nonequilibrium
thermodynamics, as the Church-Turing thesis in the theory of computability. The
primitive macroscopically definable ensembles are described. These are
ensembles with macroscopically prepared initial states. The method for
computing trajectories of primitive macroscopically definable nonequilibrium ensembles is elaborated. These trajectories
are represented as sequences of deformed equilibrium ensembles and simple
quadratic models between them. The primitive macroscopically definable
ensembles form the manifold in the space of ensembles. We call this manifold
the film of nonequilibrium states. The equation for
the film and the equation for the ensemble motion on the film are written down.
The notion of the invariant film of non-equilibrium states, and the method of
its approximate construction transform the the
problem of nonequilibrium kinetics into a series of
problems of equilibrium statistical physics. The developed methods allow us to
solve the problem of macro-kinetics even when there are no autonomous equations
of macro-kinetics

Iliya V. Karlin, Larisa L. Tatarinova, Alexander
N. Gorban, Hans Christian Ottinger

**Irreversibility in the short memory
approximation** Physica A, 327, 2003, 399-424
Online: http://arXiv.org/abs/cond-mat/0305419
v1 18 May 2003 KTGOe2003LANL.pdf

A recently introduced systematic approach to derivations of the macroscopic
dynamics from the underlying microscopic equations of motions in the
short-memory approximation [Gorban et al, Phys. Rev. E 63 , 066124 (2001)] is
presented in detail. The essence of this method is a consistent implementation
of Ehrenfest's idea of coarse-graining, realized via
a matched expansion of both the microscopic and the macroscopic motions.
Applications of this method to a derivation of the nonlinear Vlasov-Fokker-Planck equation, diffusion equation and
hydrodynamic equations of the uid with a long-range
mean field interaction are presented in full detail. The advantage of the
method is illustrated by the computation of the post-Navier-Stokes
approximation of the hydrodynamics which is shown to be stable unlike the
Burnett hydrodynamics.

Alexander N. Gorban, Iliya V. Karlin

**Family
of additive entropy functions out of thermodynamic limit**,
Physical Review E 67, 016104, 2003. Online: http://arXiv.org/abs/cond-mat/0205511
24 May 2002. PRE162003.pdf

We derive a one-parametric family of entropy functions
that respect the additivity condition, and which describe effects of finiteness
of statistical systems, in particular, distribution functions with long tails.
This one-parametric family is different from the Tsallis
entropies, and is a convex combination of the Boltzmann- Gibbs-Shannon entropy
and the entropy function proposed by Burg. An example of how longer tails are
described within the present approach is worked out for the canonical ensemble.
We also discuss a possible origin of a hidden statistical dependence, and give
explicit recipes on how to construct corresponding generalizations
of the master equation.

Gorban A. N., Karlin I. V.,

**Reconstruction Lemma and
Fluctuation-Dissipation Theorem****, **Revista Mexicana De F´isica 48
Suplemento 1, Septiembre
2002, 238 – 242. Mexico_48_1_238.pdf

We
discuss a new approach to nonequilibrium statistical
thermodynamics based on mappings of the microscopic dynamics into the
macroscopic dynamics. Near stationary solutions, this mapping results in a
compact formula for the macroscopic vector field without a hypothesis of a
separation of time scales. Relations of this formula to short-memory
approximation, the Green-Kubo formula, and expressions of transport
coefficients in terms of Lyapunov exponents are
discussed.

*Keywords: *Nonequilibrium statical
mechanics, coarse-graining, exact fluctuation-dissipation relation

Gorban A. N., Karlin I. V.

**Geometry
of irreversibility**, in: Recent Developments in Mathematical and
Experimental Physics, Volume C: Hydrodynamics and Dynamical Systems, Ed. F.
Uribe (Kluwer, Dordrecht, 2002), pp. 19-43. GeoNeo02.pdf

A general geometrical setting of nonequilibrium
thermodynamics is developed. The approach is based on the notion of the natural
projection which generalizes Ehrenfests'
coarse-graining. It is demonstrated how derivations of irreversible macroscopic
dynamics from the microscopic theories can be addressed through a study of
stability of quasiequilibrium manifolds.

A. Gorban, A. Rossiev,
N. Makarenko, Y. Kuandykov, V. Dergachev

**Recovering
data gaps through neural network methods**, International
Journal of Geomagnetism and Aeronomy vol. 3, no. 2,
pages 191-197, December 2002 geomag02.pdf

A new method is presented to recover the lost data in geophysical time series.
It is clear that gaps in data are a substantial problem in obtaining correct
outcomes about phenomenon in time series processing. Moreover, using the data
with irregular coarse steps results in the loss of prime information during
analysis. We suggest an approach to solving these problems,
that is based on the idea of modeling the data with
the help of small-dimension manifolds, and it is implemented with the
help of a neural network. We use this approach on real data and show its proper
use for analyzing time series of cosmogenic
isotopes. In addition, multifractal analysis was applied to the recovered 14C
concentration in the Earth's atmosphere.

Gorban A.N., Karlin I.V.

**Methods of nonlinear kinetics**, Contribution to the "Encyclopedia of Life Support Systems" (EOLSS
Publishers, Oxford). encboltz02.pdf
E-print: http://arxiv.org/abs/cond-mat/0306062

Nonlinear kinetic equations are reviewed for a wide
audience of specialists and postgraduate students in physics, mathematical
physics, material science, chemical engineering and interdisciplinary research.

Contents:

1. The Boltzmann equation

2. Phenomenology of the Boltzmann equation

3. Kinetic models

4. Methods of reduced description

4.1. The Hilbert method

4.2. The Chapman-Enskog method

4.3. The Grad moment method

4.4. Special approximations

4.5. The method of invariant manifold

4.6. Quasi-equilibrium approximations

5. Discrete velocity models

6. Direct simulation

7. Lattice Gas and Lattice Boltzmann models

8. Other kinetic equations

8.1. The Enskog equation for hard spheres

8.2. The Vlasov equation

8.3. The Smoluchowski equation

Gorban A.N., Karlin I.V.

**Method
of invariant manifold for chemical kinetics**** **Online:
http://arXiv.org/abs/cond-mat/0207231
v1 9 Jul 2002 InvManLANL2002.pdf

In this paper, we review the construction of low-dimensional manifolds of
reduced description for equations of chemical kinetics from the standpoint of
the method of invariant manifold (MIM). MIM is based on a formulation of the
condition of invariance as an equation, and its solution by Newton iterations.
A review of existing alternative methods is extended by a thermodynamically
consistent version of the method of intrinsic low-dimensional manifolds. A
grid-based version of MIM is developed, and model extensions of low-dimensional
dynamics are described. Generalizations to open systems are suggested. The set
of methods covered makes it possible to effectively reduce description in
chemical kinetics.

Karlin I.V., Gorban A.N.

**Hydrodynamics
from Grad's equations: What can we learn from exact solutions?****
**Annalen der Physics, 2002. Online: http://arXiv.org/abs/cond-mat/0209560 v1 24 Sep 2002. annphys02.pdf

A detailed treatment of the classical Chapman-Enskog
derivation of hydrodynamics is given in the framework of Grad's moment
equations. Grad's systems are considered as the minimal kinetic models where
the Chapman-Enskog method can be studied exactly,
thereby providing the basis to compare various approximations in extending the
hydrodynamic description beyond the Navier-Stokes
approximation. Various techniques, such as the method of partial summation, Pad_e approximants, and invariance principle are compared
both in linear and nonlinear situations.

Karlin I.V., Grmela M., Gorban A.N.

**Duality
in nonextensive statistical mechanics****.
**Physical Review E, 2002, Volume 65, 036128. P.1-4. PRE362002.pdf

We revisit recent derivations of kinetic equations based on Tsallis’
entropy concept. The method of kinetic functions is introduced as a standard
tool for extensions of classical kinetic equations in the framework of Tsallis’ statistical mechanics. Our analysis of the
Boltzmann equation demonstrates a remarkable relation between thermodynamics
and kinetics caused by the deformation of macroscopic observables.

Gorban A.N., Karlin I.V., Ottinger H.C.

**The additive generalization of the Boltzmann
entropy, **Physical Review E, 2003,
Volume 67, 067104,. Online:
http://arXiv.org/abs/cond-mat/0209319 v1 13 Sep 2002 ProofMS2003.pdf

There exists only one generalization of the classical Boltzmann-Gibbs-Shannon
entropy functional to a one-parametric family of additive entropy functionals. We find analytical solution to the
corresponding extension of the classical ensembles, and discuss in some detail
the example of the deformation of the uncorrelated state.

Gorban A.N., Karlin I.V.

**Macroscopic
dynamics through coarse-graining: A solvable example****,**
Physical Review E, 2002, Volume 65, 026116, P.1-5. PREEhr02.pdf

The recently derived fluctuation-dissipation formula (A. N. Gorban et al.,
Phys. Rev. E 63, 066124. 2001) is illustrated by the explicit computation for
McKean’s kinetic model (H. P. McKean, J. Math. Phys. 8, 547. 1967). It is
demonstrated that the result is identical, on the one hand, to the sum of the
Chapman-Enskog expansion, and, on the other hand, to
the exact solution of the invariance equation. The equality between all three
results holds up to the crossover from the hydrodynamic to the kinetic domain.

Gorban' A., Braverman M., and Silantyev
V.

**Modified Kirchhoff flow with a partially penetrable
obstacle and its application to the efficiency of free flow turbines****,** Mathematical and Computer Modelling, Volume 35, Issue
13, June 2002, P. 1371-1375. MCM2002-2.pdf

An explicitly solvable analog of the Kirchhoff flow
for the case of a semipenetrable obstacle is
considered. Its application to estimating the efficiency of free flow turbines
is discussed.

Gorban' A., Silantyev V.

**Riabouchinsky flow with
partially penetrable obstacle**, Mathematical and
Computer Modelling, Volume 35, Issue 13, June 2002, P. 1365-1370. MCM2002-1.pdf

An explicitly solvable Riabouchinsky model with a
partially penetrable obstacle is introduced. This model applied to the
estimation of the efficiency of free flow turbines allows us to take into
account the pressure drop past the lamina.

Gorban' A.N., Gorlov
A.N., Silantyev V.M.

**Limits
of the Turbine Efficiency for Free Fluid Flow**,
Journal of Energy Resources Technology - December 2001 - Volume 123, Issue 4,
pp. 311-317. Gorlov2001.pdf

An accurate estimate of the theoretical power limit of turbines in free fluid
flows is important because of growing interest in the development of wind power
and zero-head water power resources. The latter includes the huge kinetic
energy of ocean currents, tidal streams, and rivers without dams. Knowledge of
turbine efficiency limits helps to optimize design of hydro and wind power
farms. An explicitly solvable new mathematical model for estimating the maximum
efficiency of turbines in a free (nonducted) fluid is
presented. This result can be used for hydropower turbines where construction
of dams is impossible (in oceans) or undesirable (in rivers), as well as for
wind power farms. The model deals with a finite two-dimensional, partially
penetrable plate in an incompressible fluid. It is nearly ideal for
two-dimensional propellers and less suitable for three-dimensional cross-flow Darrieus and helical turbines. The most interesting finding
of our analysis is that the maximum efficiency of the plane propeller is about
30 percent for free fluids. This is in a sharp contrast to the 60 percent given
by the Betz limit, commonly used now for decades. It is shown that the Betz
overestimate results from neglecting the curvature of the fluid streams. We
also show that the three-dimensional helical turbine is more efficient than the
two-dimensional propeller, at least in water applications. Moreover,
well-documented tests have shown that the helical turbine has an efficiency of
35 percent, making it preferable for use in free water currents.

Gorban A.N., Zinovyev A.Yu.

**Visualization of
Data by Method of Elastic Maps and its Applications in Genomics, Economics and
Sociology****, **Institut des Hautes Etudes Scientifiques
Preprint. IHES M/01/36. Online: http://www.ihes.fr/PREPRINTS/M01/Resu/resu-M01-36.html
elmap.pdf

Technology of data visualization and data modeling is
suggested. The basic of the technology is original idea of elastic net and
methods of its construction and application. A short review of relevant methods
has been made. The methods proposed are illustrated by applying them to the
real biological, economical, sociological datasets and to some model data
distributions.

Gorban A.N., Karlin I.V., Ilg P., Ottinger H.C.

**Corrections
and enhancements of quasi-equilibrium states****, **J.
Non-Newtonian Fluid Mech. 2001, 96, P. 203-219. NonNew01.pdf

We give a compact non-technical presentation of two
basic principles for reducing the description of nonequilibrium
systems based on the quasi-equilibrium approximation. These two principles are:
construction of invariant manifolds for the dissipative microscopic dynamics,
and coarse-graining for the entropy-conserving microscopic dynamics. Two new
results are presented: first, an application of the invariance principle to
hybridization of micro-macro integration schemes is introduced, and is
illustrated with non-linear dumbbell models; second, Ehrenfest’s
coarse-graining is extended to general quasi-equilibrium approximations, which
gives the simplest way to derive dissipative equations from the Liouville equation in the short memory approximation.

Gorban A.N., Karlin I.V., Ottinger H.C., Tatarinova L.L.

**Ehrenfest****’ argument extended to a formalism of nonequilibrium thermodynamics,**
Physical Review E, 2001. Volume 63, 066124, P.1-6. PREEhr01.pdf

A general method of constructing dissipative equations
is developed, following Ehrenfest’sidea of coarse
graining. The approach resolves the major issue of discrete time coarse
graining versus continuous time macroscopic equations. Proof of the H theorem
for macroscopic equations is given, several examples supporting the
construction are presented, and generalizations are suggested.

Gorban A.N., Zinovyev A.Yu., Popova T.G.

**Self-organizing
approach for automated gene identification in whole genomes****,**
Institut des Hautes Etudes Scientifiques Preprint. IHES. December 12, 2001, Online: http://arXiv.org/abs/physics/0108016
v1 10 Aug 2001 lanlgpz01.pdf

An approach based on using the idea of distinguished coding phase in explicit
form for identi cation of protein-coding regions in
whole genome has been proposed. For several genomes an optimal window length
for averaging GC-content function and calculating codon frequencies has been
found. Self-training procedure based on clustering in multidimensional space of
triplet frequencies is proposed.

Gorban A.N., Zinovyev A.Yu., Popova
T.G.

**Statistical approaches to automated gene identification
without teacher. **Institut des Hautes Etudes Scientifiques
Preprint. IHES M/01/34. Online: http://www.ihes.fr/PREPRINTS/M01/Resu/resu-M01-34.html
geneid.pdf

Overview of statistical methods of gene identification is made. Particular
attention is given to the methods which need not a training set of already
known genes. After analysis several statistical approaches are proposed for
computational exon identification in whole genomes. For several genomes an
optimal window length for averaging GC-content function and calculating codon
frequencies has been found. Self-training procedure based on clustering in
multidimensional codon frequencies space is proposed.

A. N. Gorban, K. O. Gorbunova, D. C. Wunsch II

**Liquid
Brain: Kinetic Model of Structureless Parallelism,****
**liquidbrain.pdf

A new formal model of parallel computations, the Kirdin
kinetic machine, is suggested. It is expected that this model will play the
role for parallel computations similar to Markov normal algorithms, Kolmogorov
and Turing machine or Post schemes for sequential computations. The basic ways
in which computations are realized are described; correctness of the elementary
programs for the Kirdin kinetic machine is
investigated. It is proved that the determined Kirdin
kinetic machine is an effective calculator. A simple application of the Kirdin kinetic machine, heap encoding, is suggested.
Subprograms similar to usual programming enlarge the Kirdin
kinetic machine.

Gorban A.N., Karlin I.V., Zmievskii V.B., Dymova S.V.

**Reduced
description in the reaction kinetics****, **Physica A,
2000, 275, P.361-379. GKZD2000.pdf

Models of complex reactions in thermodynamically isolated systems often
demonstrate evolution towards low-dimensional manifolds in the phase space. For
this class of models, we suggest a direct method to construct such manifolds,
and thereby to reduce the effective dimension of the problem. The approach
realizes the invariance principle of the reduced description, it is based on
iterations rather than on a small parameter expansion, it leads to tractable
linear problems, and is consistent with thermodynamic requirements. The
approach is tested with a model of catalytic reaction.

Gorban A.N., Popova
T.G., Sadovsky M.G.

**Classification
Of Symbol Sequences Over Thier
Frequency Dictionaries: Towards The Connection Between Structure And Natural
Taxonomy****,** Open Sys. & Information Dyn. 7: 1-17, 2000. opsygps00.pdf

The classifications of bacterial 16S RNA sequences developed over the real and
transformed frequency dictionaries have been studied. Two sequences considered
to be close each other, when their frequency dictionaries were close in
Euclidean metrics. A procedure to transform a dictionary is proposed that makes
clear some features of the information pattern of a symbol sequence. A
comparative study of classifications developed over the real frequency
dictionaries vs. the transformed ones has been carried out. A correlation
between an information pattern of nucleotide sequences and taxonomy of the
bearer of the sequence was found. The sites with high information value are
found, that were the main factors of the difference between the classes in a
classification. The classification of nucleotide sequences developed over the
real frequency dictionaries of the thickness 3 reveals the best correlation to
a gender of bacteria. A set of sequences of the same gender is included
entirely into one class, as a rule, and the exclusions occur rarely. A hierarchical
classification yields one or two taxonomy groups on each level of the
classification. An unexpectedly often (in comparison to the expected), or
unexpectedly rare occurrence of some sites within a sequence makes a basic
difference between the structure patterns of the classes yielded; a number of
those sites is not too great. Further investigations are necessary in order to
compare the sites revealed with those determined due to other methodology.

A. **N.
**Gorban, I.V. Karlin, and V.B. Zmievskii

**Two-Step Approximation of
Space-Independent Relaxation****,** TRANSPORT THEORY *AND *STATISTICAL
PHYSICS, 28(3) (1999), 271-296. GorKarZmiTTSP99.pdf

In this paper we introduce a new method of constructing approximate trajectories for space independent kinetic equations confirming to the second law of thermodynamics. Classical examples are the space independent Boltzmann equation and chemical kinetics equations for closed homogeneous systems. This family of kinetic equations is characterized by the following general properties:

(1). There exists a set of functions which remain constant on a solution (these are density, momentum and energy in context of the Boltzmann equation).

(ii). There exists a convex function which monotonically decreases along any solution from its value in the initial state to an absolute minima in the final equilibrium state (this is the H-theorem for the Boltzmann equation) .

Usually we do know only the initial and the final (equilibrium) states, and the kinetic equation neither can be solved exactly, nor contains small parameters to develop a reliable perturbation theory. Still, we would like to get (perhaps a rather rough but a simple) approximation of the relaxation trajectory.

** **

An** **express method to
approximate trajectories of space independent kinetic equations is developed.
It involves a two-step treatment of relaxation through a quasiequilibria
located on a line emerging from the initial state in the direction prescribed
by the kinetic equation. **A **test for the Boltzmann equation shows the
validity of the method.

A.N. Gorban, A.A. Rossiev,
D. C. Wunsch II

**Neural Network Modeling
of Data with Gaps: Method of Principal Curves, Carleman's Formula, and Other****, **The
talk was given at the USA-NIS Neurocomputing
opportunities workshop, Washington DC, July 1999 (Associated with IJCNN'99).

Online: http://arXiv.org/abs/cond-mat/0305508
21 May 2003 gaps.pdf

A method of modeling data with gaps by a sequence of
curves has been developed. The new method is a generalization of iterative
construction of singular expansion of matrices with gaps. Under discussion are
three versions of the method featuring clear physical interpretation:

1) linear: modeling the data by a sequence of linear
manifolds of small dimension;

2) quasilinear: constructing "principal curves": (or "principal
surfaces"), univalently projected on the linear
principal components;

3) essentially non-linear, based on constructing "principal curves":
(principal strings and beams) employing the variation principle; the iteration
implementation of this method is close to Kohonen
self-organizing maps.

The derived dependencies are extrapolated by Carleman’
formulas. The method is interpreted as a construction of neural network
conveyor designed to solve the following problems:

1) to fill gaps in data;

2) to repair data, to correct initial data values in such a way as to make the
constructed models work best;

3) to construct a calculator to fill gaps in the data line fed to the input.

Gorban A. N.

**Neuroinformatics****: What are us, where are we
going, how to measure our way****? **The lecture was given at the USA-NIS Neurocomputing opportunities workshop, Washington
DC, July 1999 (Associated with IJCNN'99) neurolec.pdf

What is neuroinformatics?
For me here and now neuroinformatics is a direction
of science and information technology, dealing with development and study of
the methods for solution of problems by means of neural networks. A base
example of artificial neural network, which will be referred to below, is a
feed-forward network from standard neurons.

Alexander N. Gorban, Eugeniy
M. Mirkes and Victor G. Tsaregorodtsev

**Generation of
Explicit Knowledge from Empirical Data through Pruning of Trainable Neural
Networks****,** International Joint Conference on Neural
Networks, Washington, DC July 10-16, 1999. know.pdf
E-print: http://arxiv.org/abs/cond-mat/0307083

This paper presents a generalized technology of
extraction of explicit knowledge from data. The main ideas are:

1) maximal reduction of network complexity (not only removal of neurons or
synapses, but removal all the unnecessary elements and signals and reduction of
the complexity of elements),

2) using of adjustable and flexible pruning process (the pruning sequence
shouldn't be predetermined - the user should have a possibility to prune
network on his own way in order to achieve a desired network structure for the
purpose of extraction of rules of desired type and form),

3) extraction of rules not in predetermined but any desired form.

Some considerations and notes about network architecture and training process
and applicability of currently developed pruning techniques and rule extraction
algorithms are discussed. This technology, being developed by us for more than
10 years, allowed us to create dozens of knowledge-based expert systems.

A.
N. Gorban, I. V. Karlin

**Schrodinger operator in an overfull set**, Europhys. Lett., 42 (2) (1998),
113-117. GK98Shro.pdf

Operational simplicity of an expansion of a wave function over a basis in the
Hilbert space is an undisputable advantage for many non-relativistic
quantum-mechanical computations. However, in certain cases, there are several
\natural" bases at one's disposal, and it is not easy to decide which is
preferable. Hence, it sounds attractive to use several bases simultaneously,
and to represent states as expansions over an overfull set obtained by a
junction of their elements. Unfortunately, as is well known, such a
representation is not unique, and lacks many convenient properties of full sets
(e.g., explicit formulae to compute coeffcients of
expansions). Because of this objection, overfull sets are used less frequently
than they, perhaps, deserve. We introduce a variational
principle which eliminates this ambiguity, and results in an expansion which
provides “the best" representation to a given Schrodinger operator.

A.N. Gorban, D.C. Wunsch II

**The**** General
Approximation Theorem.**
In Proceedings IJCNN'98, IEEE, 1998. PP. 1271-1274.

A general approximation theorem is proved. It uniformly envelopes both the
classical Stone theorem and approximation of functions of several variables by
means of superpositions and linear combinations of
functions of one variable. This theorem is interpreted as a statement on
universal approximating possibilities ("approximating omnipotence")
of arbitrary nonlinearity. For the neural networks, our result states that the
function of neuron activation must be nonlinear - and nothing else. The second
theorem states the possibility of exact representation of all polynomials of
several variables by means of arbitrary nonlinear polynomial of one variable,
linear functions and superposition operations.

N. N. Bugaenko,
A. N. Gorban and M. G. Sadovsky,

**Maximum Entropy Method in Analysis of Genetic Text and Measurement of its Information Content****,** Open Systems
& Information Dynamics, 1998, Volume 5, Number 3, Pages 265-278.

The information capacity in frequency dictionaries of nucleotide sequences is
estimated through the efficiency of reconstruction of a longer frequency
dictionary from a short one. This reconstruction is performed by the maximum
entropy method. Real nucleotide sequences are compared to random ones (with the
same nucleotide composition). Phages genes from NCBl bank were analyzed. THe significant difference of real genetic text from random
sequences is observed for the dictionary length *q*=2,5 and 6.

Karlin
I.V., Gorban A.N., Dukek G., Nonnenmacher
T. F.

**Dynamic
correction to moment approximations.** Physical Review E, February 1998 Volume 57, Number
2, P.1668-1672. KGDN98.pdf

Considering the Grad moment ansatz as a suitable first approximation to a
closed finite-moment dynamics, the correction is derived from the Boltzmann
equation. The correction consists of two parts, local and nonlocal. Locally
corrected thirteen-moment equations are demonstrated to contain exact transport
coefficients. Equations resulting from the nonlocal correction give a
microscopic justification to some phenomenological theories of extended
hydrodynamics.

Gorban
A. N.

**Approximation
of Continuos Functions of Several Variables by an
Arbitrary Nonlinear Continuous Function of One Variable, Linear Functions, and
Their Superpositions,**
Appl. Math. Lett., Vol. 11, No. 3, pp 45-49,
1998 approx98.pdf

Linear spaces of continuous functions of real variables closed under the superposition operation are considered. It has been proved that when such a space contains constants, linear functions, and at least one nonlinear function, it is dense in the space of all continuous functions in the topology of uniform convergence on compact sets. So, the approximation of continuous functions of several variables by an arbitrary nonlinear continuous function of one variable, linear functions, and their superpositions is possible.

Karlin I.V., Gorban A.N., Succi S., Boffi V.

**Maximum
Entropy Principle for Lattice Kinetic Equations.**
Physical Review Letters Volume 81, Number 1, 6 July 1998, P.6-9. p6_11998.pdf

The entropy maximum approach to constructing equilibria in lattice kinetic
equations is revisited. For a suitable entropy function, we derive explicitly
the hydrodynamic local equilibrium, prove the H theorem for lattice Bhatnagar-Gross-Krook models, and
develop a systematic method to account for additional constraints.

Gorban A.N., Shokin
Yu.I., Verbitskii V.I.

**Simultaneously
dissipative operators and the infinitesimal wrapping effect in interval spaces****,
**Computational Technologies, 2 (4) (1997), 16-48.**
** Online: http://arXiv.org/abs/physics/9702021
, 1997. GorbanShokVerVychTechnol.pdf

We study simultaneously dissipative linear operators.
The family of linear operators is simultaneously dissipative, if there exists a
norm relative to which all the operators are dissipative. We construct various
sufficient conditions for existence of such a norm. We consider two examples of
applications for this theory: stability of chemical kinetics and phenomenon of
interval expansion.

In solving a system of ordinary differential equations by an interval method
the approximate solution at any considered moment of time t represents a set
(called interval) containing the exact solution at the moment t. The intervals
determining the solution of a system are often expanded in the course of time
irrespective of the method and step used.

The phenomenon of interval expansion, called the wrapping or Moore sweep effect, essentially decreases the
efficiency of interval methods. In the present work the notions of the interval
and the Moore effect are formalized and the Infinitesimal Moore (wrapping) Effect (IME) is studied for
autonomous systems on positively invariant convex compact. With IME the
intervals expand along any trajectory for any small step, and that means that
when solving a system by a stepwise interval numerical method with any small
step the interval expansion takes place for any initial data irrespective of
the applied method. The local conditions of absence of IME in terms of Jacoby
matrices field of the system are obtained. The relation between the absence of
IME and simultaneous dissipativity of the Jacoby
matrices is established, and some sufficient conditions of simultaneous dissipativity are obtained.

M.Yu. Senashova, A.N. Gorban, D. C. Wunsch II

**Back-propagation
of accuracy,** The talk given on ICNN97 (The 1997 IEEE
International Conference on Neural Networks, Houston, USA), Online: http://arXiv.org/abs/cond-mat/0305527
gorsenwu.pdf

In this paper we solve the problem: how to determine maximal allowable errors,
possible for signals and parameters of each element of a network proceeding
from the condition that the vector of output signals of the network should be
calculated with given accuracy? "Back-propagation of accuracy" is
developed to solve this problem.

A. N: Gorban, Ye. M. Mirkes, D.C. Wunsch
II

**High order
ortogonal tensor networks: information capacity and
reliability****. **The talk given on ICNN97 (The 1997 IEEE International Conference on Neural Networks, Houston,
USA), gomirwu1.pdf

Neural networks based on construction of ortogonal
projectors in the tensor power of space of signals are described. A sharp
estimate of their ultimate information capacity is obtained. The numbers of
stored prototype patterns (prototypes) can many times exceed the number of
neurons. A comparison with the error control codes is made.

Gorban A.N., Karlin I.V.

**Short-Wave
Limit of Hydrodynamics: A Soluble Example****.** Physical
Review Letters, Volume 77, Number 2, 8 July 1996. P. 282-285. p282_11996.pdf

The Chapman-Enskog series for shear stress is summed
up in a closed form for a simple model of Grad moment equations. The resulting
linear hydrodynamics is demonstrated to be stable for all wavelengths, and the
exact asymptotic of the acoustic spectrum in the short-wave domain is
obtained.

Gorban A.N., Karlin I.V. Nonnenmacher T. F., Zmievskii
V.B.

**Relaxation
Trajectories: Global approximation****.** Physica A,
1996, 231, P.648-672. GKZNPhA96.pdf

Gorban A. N., Karlin I. V.

**Scattering
rates versus moments: Alternative Grad equations,**
Physical Review E October 1996 Volume 54, Number 4, P. 3109-3112. pR3109_11996.pdf

Scattering rates (moments of collision integral) are treated as independent
variables, and as an alternative to moments of the distribution function, to
describe the rarefied gas near local equilibrium. A version of the entropy
maximum principle is used to derive the Grad-like description in terms of a
finite number of scattering rates. The equations are compared to the Grad
moment system in the heat nonconductive case. Estimations for hard spheres
demonstrate, in particular, some 10% excess of the viscosity coefficient
resulting from the scattering rate description, as compared to the Grad moment
estimation.

Gorban A. N., Karlin I. V.

**On “Solid
Liquid” limit of Hydrodynamic Equations,** Transport Theory and
Statistical Physics 24 (9) (1995), 1419-1421. GKSolJet95s.pdf

An “infinitely viscid threshold” for compressible liquid is described. A rapid
compression of a flux amounts to a strong deceleration of particles (particles
loose velocity comparable to heat velocity on a distance compatible to the main
free path). Such a strong deceleration is described in the frames of
hydrodynamic equations by a divergency of viscosity.
A fluid becomes “solid”.

A.N. Gorban, C. Waxman,

**Neural Networks for
Political Forecast.*** Proceedings of
the 1995 World Congress On Neural Networks*, A Volume in the INNS Series of
Texts, Monographs, and Proceedings, Vol. 1, 1995. (A preliminary** 1992** publication of Cory Waxman, the
student of A.Gorban, is available in electronic form
– see below)

Cory Waxman,

**The
History of US Presidential Elections from Siberian NC Point of View**,
In: Neuroinformatics and Neurocomputers, 7-10 Oct
1992, Rostov-on-Don, Russia, Proc. RNNS/IEEE Symposium, vol.2, pp. 1000 – 1010,
IEEE press, 1992. Cory.pdf http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=00268530

Tests were performed with the program "US Presidential Elections" and the future relationship between neurocomputers and the human sciences was discussed.

This paper will discuss the type of neurocomputer being developed in Krasnoyarsk (by S. E. Gilev, A. N. Gorban, E. M. Mirkes), describe the results of some experiments, and conclude with a discussion on possible future applications of neurocomputers in the human sciences.

Perhaps
the most revolutionary aspect of neurocomputers is that they can **be **applied
to problems of which we have very little understanding. This is quite different
than the standard use of computers in science. Often scientists apply computers
to algorithmic problems (in which the problem can be solved by a predefined
series of steps). **For **such problems traditional computers are of
tremendous value, and can work thousands of times faster than humans. But there
is another area of science where the exact nature or form of the problem is
rarely well understood – the human sciences. In History, Political Science,
Psychology, and Education sciences there are many possible applications of **NC’s.
**We have already discussed some direct applications in history and political
science. We also saw how new questions might be formed in the course of these
applications. T his ability to find new questions
should not be overlooked as it has been said that sometimes the question is
much more important than the answer.

Dorrer, M.G., Gorban, A.N., Zenkin, V.I.

**Neural
networks in psychology: classical explicit diagnoses**, In: Neuroinformatics and Neurocomputers, 1995, Second
International Symposium, 20-23 Sep 1995, Rostov on Don, Russia, pp. 281-284,
DOI: 10.1109/ISNINC.1995.480869

The purpose of this work is to employ trainable neural networks to start solving the problem facing the designers and users of computer psychological tests: cultural, national and social adaptation of tests. Mathematical construction of up-to-date objective diagnostic tests is based on a comparison of the revealed condition with the norm standard. It is understandable that the norms worked out for one socio-cultural group are not necessarily the same for the other. By way of example it is possible to cite the difficulties to be reckoned with in adapting foreign techniques. Neural networks have been successfully used for classical explicit diagnoses. A typical experiment is described

Alexander N. Gorban, Iliya
V. Karlin

**Method of invariant manifolds and
regularization of acoustic spectra****,** Transport Theory and Statistical
Physics 23 (5) (1994), 559-632. GorbanKarlinTTSP94.pdf

A new approach to the problem of reduced description for Boltzmann-type systems
is developed. It involves a direct solution of two main problems: thermodynamicity and dynamic invariance of reduced
description. A universal construction is introduced, which gives a
thermodynamic parameterization of an almost arbitrary approximation.
Newton-type procedures of successive approximations are developed which correct
dynamic noninvariance. The method is applied to
obtain corrections to the local Maxwell manifold using parametrics
expansion instead of Taylor series into powers of Knudsen number. In
particular, the high frequency acoustic spectrum is obtained.

Alexander N. Gorban', Iliya
V. Karlin

**General
approach to constructing models of the Boltzmann equation****,**
Physica A, 1994, 206, P.401-420. GKPhA94.pdf

The problem of thermodynamic parameterization of an arbitrary approximation of
reduced description is solved. On the base of this solution a new class of
model kinetic equations is constructed that gives a model extension of the
chosen approximation to a kinetic model. Model equations describe two
processes: rapid relaxation to the chosen approximation along the planes of rapid
motions, and the slow motion caused by the chosen approximation. The H-theorem
is proved for these models. It is shown, that the rapid process always leads to
entropy growth, and also a neighborhood of the
approximation is determined inside which the slow process satisfies the
H-theorem. Kinetic models for Grad moment approximations and for the
Tamm-Mott-Smith approximation are constructed explicitly. In particular, the
problem of concordance of the ES-model with the H-theorem is solved.

A.N.
Gorban, I. **V. **Karlin,

**Nonarbitrary regularization of acoustic spectra**,
Transport Theory and Statistical Physics, 22(1), 121-124.

We
suggest a method of constructing dynamic invariant manifolds for the Boltzmann
equation. It aims to improve the Chapman-Enskog
expansion (CE) free of ad hoc assumptions. The problems of the CE method are
well known, for example, a short-wave instability of the Burnett approximation**.
**Many attempts were made to improve the CE expansion. In particular, in our
previous work we used the idea of partial summing**. **However, all these
attempts have an ad hoc character. The famous KAM theory serves us as a
prototype. In KAM, the rapidly converging Newton method is used instead of
diverging Taylor expansion, and one searches for an invariant manifold rather
than for a solution. Following ides of KAM, we use the Newton method. Each
iteration is concordant with the *H*-theorem.

Our method consists of two main parts:

1. Construction of a special thermodynamic parameterization for an arbitrary
manifold which gives dynamic

equations on this manifold (this part has no analogue in KAM
and it is caused by the necessity to satisfy the H-theorem at every step).

2. Correction of the dynamic noninvariance of a manifold by the Newton method.

We describe the method for a general dynamic system with a global convex
H-function.

Alexander N. Gorban' , Iliya V. Karlin

**Thermodynamic
parameterization****,** Physica A,
1992, 190, P.393-404 GKPhA92.pdf

A new method of successive construction of a solution is developed for problems
of strongly nonequilibrium Boltzmann kinetics beyond
normal solutions. Firstly, the method provides dynamic equations for any
manifold of distributions where one looks for an approximate solution.
Secondly, it gives a successive procedure of obtaining corrections to these
approximations. The method requires neither small parameters, nor strong
restrictions upon the initial approximation; it involves solutions of linear
problems. It is concordant with the H-theorem at every step. In particular, for
the Tamm-Mott-Smith approximation, dynamic equations are obtained, an expansion
for the strong shock is introduced, and a linear equation for the first
correction is found.

Alexander
N. Gorban', Iliya V. Karlin

**Structure
and approximations of the
Chapman-Enskog expansion for the linearized Grad
Equations**, Transport Theory and
Statistical Physics, 21(1&2), 101-117 (1992).

A detailed structure of the Chapman-Enskog
expansion for the linearized Grad moment equations is determined. A method of
partial summing of the Chapman-Enskog series is
introduced, and is used to remove short-wave instability of the Burnett
approximations.

Gilev, S.Y., Gorban, A.N., Mirkes, Y.M.,

**Internal conflicts in neural
networks,** In: Neuroinformatics and
Neurocomputers, 1992., RNNS/IEEE Symposium, Vol. 1,
pp. 219-225. DOI: 10.1109/RNNS.1992.268591

Hierarchical neural networks consisting of small expert-networks are considered. Algorithms of fast parallel learning are proposed. The approach proposed greatly enlarges the information capacity of the network and accelerates learning.

V. I. Verbitskii and A. N. Gorban'

**Jointly
dissipative operators and their applications, **Siberian Mathematical
Journal, Volume 33, Number 1 (1992), 19-23, DOI: 10.1007/BF00972932

The jointly dissipative operators were introduced by Verbitskii
and Gorban' (1989). Let *E*
be an *n*-dimensional real or complex
linear space, and let *L*(*E*) be the space of linear operators in *E*. Let us introduce a norm ||…|| on *E* and the corresponding norm in *L*(*E*). An operator *A *from* L(E) *is
said to be *dissipative* if ||exp(*tA*)||≤1
for all *t*≥0. It is *stable dissipative *(in the paper due to
the interpreter mistake a term “*roughly
dissipative*” is used) if there is ε > 0 such that ||exp(tA)||≤exp(-ε*t*) for all *t*≥0.
For the existence of a norm with respect to which the operator A is be roughly
dissipative it is necessary and sufficient that the system (i)
be asymptotically stable, i.e., that the matrix of A be stable (i.e., that the
spectrum of A lie in the open left halfplane). A
family of operators is said to be ** jointly dissipative** (resp. jointly
roughly dissipative) if there exists a norm with respect to which all operators
from this family are dissipative (resp., roughly dissipative). The jointly dissipative operators find application
in the analysis of dynamical properties of nonlinear systems of ordinary
differential equations and in some applications (chemical kinetics, numerical
analysis). In the present paper we discuss the properties of jointly
dissipative operators and some of their applications. For example, the
following theorems are proved: (Theorem 1) Suppose the
family

**1991 **

N. N. Bugaenko, A. N. Gorban', and I. V. Karlin

**Universal
expansion of three-particle distribution function,** Theoretical
and Mathematical Physics, Vol. 88,
No. 3, 1991. Translated from Teoreticheskaya i Matematicheskaya Fizika, Vol. 88, No. 3, pp. 430-441, September, 1991.TMF1990.pdf

A universal, i.e., not dependent on the Hamiltonian of the two-particle
interaction, expansion of the equilibrium three-particle distribution function
with respect to the two-particle correlation functions is constructed. A
diagram technique that permits systematic calculation of the coefficients of
this expansion is proposed. In particular, it is established that allowance for
the first four orders in the absence of long-range correlations gives the
Kirkwood approximation. Corrections to the Kirkwood approximation both in the
presence and absence of long-range correlations are found. Our starting point
is the construction of a quasi-equilibrium ensemble for given two-particle
distribution function

G.S.Yablonskii, V.I.Bykov, A.N. Gorban, and V.I.Elokhin

**Kinetic Models of Catalytic Reactions (Comprehensive
Chemical Kinetics, V.32,** ed. by R.G. Compton), Elsevier, Amsterdam, 1991,
396p.

Synopsis

This book has been written by a group of
mathematicians and chemists whose common interest is in the complex dynamics of
catalytic reactions. Based on developments in mathematical chemistry, a general
theory is described that allows the investigation of the relationships between
the kinetic characteristics of complex reactions and their detailed reaction
mechanism. Furthermore, a comprehensive analysis is made of some typical
mechanism of catalytic reactions, in particular for the oxidation of carbon
monoxide on platinum metals. In fact, the book presents "three
kinetics": (a) detailed, oriented to the elucidation of a detailed
reaction mechanism according to its kinetic laws; (b) applied, with the aim of
obtaining kinetic relationships for the further design of chemical reactors;
and (c) mathematical kinetics whose purpose is the analysis of mathematical
models for heterogeneous catalytic reactions taking place under steady- or
unsteady-state conditions.

**Contents**

1. Minimum minimorum. 2. The development of basic
concepts of chemical kinetics in heterogeneous catalysis. 3. Formalism of
chemical kinetics. 4. Graphs in chemical kinetics. 5. Simplest non-linear
mechanisms of catalytic reactions producing critical phenomena. 6. Studies of kinetic
models for oxidation reactions over metals (exemplified by CO oxidation). 7.
Critical retardation effects and slow relaxations. 8. Conclusions. Index.

(Review on this book: *Journal of American Chemical Society (JAChS),* V.114, n 13, 1992; sections “Reviews on the
book”, W. Henry Weinberg, review on the book "Comprehensive Chemical
Kinetics", Volume 32, Kinetic Models of Catalytic Reactions, Elsevier,
1991).

A. N. Gorban', E. M. Mirkes, A. N. Bocharov, and V. I. Bykov,

**Thermodynamic consistency of kinetic data****,** Combustion,
Explosion, and Shock Waves, Volume 25, Number 5 / September, 1989, 593-600, DOI: 10.1007/BF00772975 Consistency1989.pdf

It is well known that the rate constants of different elementary reactions are often interdependent. Relationships determined by the principle of detailed balance exist between them when microreversibility is valid and by the generalizations of that principle when it is not (for example, in magnetic fields, during macroscopic rotations, etc.). Nevertheless, in practice the verification of consistency in the kinetic constants for complicated transformation schemes involves a certain amount of technical difficulty. The problem of consistency in the kinetic constants arises especially sharply in connection with the creation of kinetic data banks intended for widespread use. Here it is impossible to avoid solving that problem or examining each multistage reaction separately, without leaving the user with the burden of finding a way to carry out this analysis. Thus, the methods for establishing the consistency of these constants, along with the conditions under which this consistency may fail, must be analyzed and suitable algorithms and programs have to be developed. We proposed such methods, developed algorithms, implemented and tested them.

Gorban A.N., Bykov
V.I.

**A model of autooscillations in association reactions****, **Chemical
Engineering Science. 1987, Vol. 42, No. 5. P. 1249-1251. BG1987.pdf

The aim of this paper is to show that association reactions can result in the
appearance of autooscillations in nonlinear systems.

Gorban A.N., Bykov
V.I., Yablonskii G.S.

**Thermodynamic
function analogue for reactions proceeding without interaction of various
substances**, Chemical Engineering Science, 1986. Vol. 41,
No. 11. P. 2739-2745. BGYa1986.pdf

Function similar to Lyapunov’s function has been
constructed for reactions with $a_i A_i \to b_j A_j$
stages. This provides for the quasi-thermodynamics of the appropriate kinetic
model, which implies steady-state uniqueness and global stability in the
reaction polyhedron. The kinetic law generalizing the Marcelin-de Donder kinetics has been written for a separate stage.
Explicit Lyapunov thermodynamic functions have been
written for various conditions of the reaction proceeding in closed systems.
The matrix of linear approximation close to equilibrium is expressed by means
of the introduced scalar product. Particularly, the absence of damped
oscillations as equilibrium is approached as shown.

V. I. Bykov, A. N. Gorban and G. S. Yablonskii,

**Description
of nonisothermal reactions in terms of Marcelin-De-Donder kinetics and its generalizations,** React. Kinet. Catal. Lett.,
Vol. 20, Nos. 3-4 (1982).

A general form for the description of nonisothermal
reactions in closed chemical systems in terms of the Marcelin-de-Donder kinetics and explicit forms of the Lyapunov functions for the systems treated under various
conditions are suggested.

V. I. Bykov,
A. N. Gorban', and T. P. Pushkareva

**Singularities
in the relaxation periods in the oxidation of CO on Platinum,** Teoreticheskaya
i Eksperimental'naya Khimiya, Vol. 18, No, 4, pp 431-439, July-August, 1982.
Original article submitted July 13, 1981. SloRelCO1982.pdf
(Translated from Russian by Plenum, in the file some of the Plenum translation
mistakes are corrected).

When studying the process dynamics of chemical reactions the first problem is
generally considered to be its limiting (for t → ∞)
conditions. But besides a reply to the question "what will
happen at t → ∞ ?" it is also important to know how rapidly the
limiting behavior is established. The slow
establishment of chemical equilibrium, associated with delays in the reaction
far from equilibrium (the induction periods) has been studied in chemistry
since the time of van't Hoff. At present, interest in
slow relaxations arises from experiments in which it was found that for certain
chemical (including heterogeneous catalytic) reactions the reactant
concentrations may slowly approach their limiting (steady-state) values,
although the observed rate of reaction may remain fairly high. Where are the
reasons of such a situation in "intrinsic" relaxation processes which
are determined directly by the reaction mechanism, or in "extrinsic"
relaxation processes arising from reasons of a non-kinetic nature (the diffusion
of the substances within the catalyst, a slow variation in its structure, etc.). Slow relaxations of a purely kinetic (intrinsic) nature
are possible. This possibility has been demonstrated for the oxidation of CO on
Pt. The surface of the singularities in the relaxation time has been
constructed for this specific catalytic oxidation reaction.

** **

Gorban A.N., Bykov
V.I.

**Macroscopic
clusters induced by diffusion in a catalytic oxidation reactions****,**
ChemicaI Engineering Science, 1980. Vol. 35, P.
2351-2352 BG1980.pdf

V. I. Elokhin, G. S. Yablonskii, A. N. Gorban and V. M. Cheresiz,

**Dynamics of chemical reactions and
nonphysical steady states**, React. Kinet. Catal. Lett., Vol. 15, No. 2
(1980), 245-250 RKCL_80_EYaGCh.pdf

Data on the position of nonphysical (lying beyond the region of determination) steady states are shown to be of use for understanding the dynamic behavior of chemical reactions, in particular, the reasons for slow relaxations. As a rule, the kinetic equations are nonlinear and should have several steady-state solutions, but not all of them are physically meaningful (negative and complex steady-state solutions are possible). But as has been shown, slow transient regimes can also be observed when the physically meaningless steady-state solutions are positioned near the reaction polyhedron.

Gorban A.N.

**Singularities of Transition Processes In
Dynamical Systems. **http://arXiv.org/abs/chao-dyn/9703010
v1 18 Mar 1997, Translation of Candidate (Ph.D)
Thesis, 1980 PhDslowrelax.pdf

The paper gives the systematic analysis of singularities of transition
processes in general dynamical systems. Dynamical systems depending on
parameter are studied. A system of relaxation times is constructed. Each relaxation
time depends on three variables: initial conditions, parameters k of the system
and accuracy \epsilon of relaxation. This system of times contains: the time
before the first entering of the motion into \epsilon -neighbourhood of the
limit set, the time of final entering in this neighbourhood and the time of
stay of the motion outside the \epsilon -neighbourhood of the limit set. The
singularities of relaxation times as functions of (x_0; k) under fixed \epsilon
are studied. A classification of different bifurcations (explosions) of limit
sets is performed. The bifurcations fall into those with appearance of new
limit points and bifurcations with appearance of new limit sets at finite
distance from the existing ones. The relations between the singularities of
relaxation times and bifurcations of limit sets are studied. The peculiarities
of dynamics which entail singularities of transition processes without
bifurcations are described as well. The peculiarities of transition processes
under perturbations are studied. It is shown that the perturbations simplify
the situation: the interrelations between the singularities of relaxation times
and other peculiarities of dynamics for general dynamical system under small
perturbations are the same as for smooth two-dimensional structural stable
systems.

Gorban
A.N.

**Invariant sets for kinetic equations, **React. Kinet. Catal.
Lett., Vol. 10, No. 2 (1979), 187-190. RKCL1978.pdf

Some sets in the space of compositions possessing an invariance property are
considered for a closed system, where a complex chemical reaction of a known
mechanism proceeds. If the vector of concentrations belongs to such a set at a
certain moment of time, it will remain within it at any succeeding moment. Some
possible applications are discussed. The most important circumstance of the
above analysis is the fact that these positively invariant sets are strongly
dependent on the detailed reaction mechanism. This may be used for discrimination
of various mechanisms under consideration.

** **

A.N. Gorban'

**Sets
of removable singularities and continuous maps,**

Sibirskii Matematicheskii Zhurnal, Vol. 19, No. 6, pp, 1388-1391, November-December, 1978. Original article submitted September 27, 1976.

We examine sets of removable singularities of analytic functionals (negligible sets) in topological vector spaces (TVS). We prove that those of them any continuous image of which also is removable can be completely described in general topology terminology (compactness, minimal cardinality of everywhere dense subset). We examine only TVS over the complex number field C.

** **

A. N. Gorban' and V. B. Melamed

**Certain
properties of Fredholm analytic sets in Banach spaces**, Sibirskii Matematicheskii Zhurnal, Vol. 17,
No. 3, pp. 682-685, May-June, 1976. Original article submitted December 9,
1974. SMZh1976.pdf

With the aid of the Lyapunov-Schmidt method of transition
to a finite-dimensional equation, we prove in this paper certain assertions
about analytic sets in complex Banach spaces. The
principal result is a counterpart of the finite-dimensional Remmert~Stein
theorem, stating that an analytic set in an open set U is either discrete , or
it contains points that are as close as desired to the boundary of U. As an
application we shall prove the nonnegativeness of the
rotation of the vector field x~Ax with an analytic
and completely continuous operator A; we also consider the finiteness of the
number of solutions of an equation that depends on a parameter.