Updated docs and renamed some stuff and moved some stuff.

doc/conf.py

@@ -52,7 +52,7 @@
 
 # -- AutoAPI configuration ---------------------------------------------------
 autoapi_dirs = ['../src']
-
+inheritance_graph_attrs = dict(rankdir="TB",)
 # -- Bibliography configuration ----------------------------------------------
 bibtex_bibfiles = ['references.bib']
 
@@ -82,7 +82,7 @@
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = None
-
+add_module_names = False
 
 # -- Options for HTML output -------------------------------------------------
 
@@ -119,6 +119,7 @@
 htmlhelp_basename = 'probabilistic_modelsdoc'
 
 
+
 # -- Options for LaTeX output ------------------------------------------------
 
 latex_elements = {

doc/distributions.rst

@@ -0,0 +1,71 @@
+#############
+Distributions
+#############
+
+At the core of each probabilistic models are known distributions. This package implements the following distributions
+
+-  :class:`probabilistic_model.distributions.distributions.SymbolicDistribution`
+-  :class:`probabilistic_model.distributions.distributions.IntegerDistribution`
+-  :class:`probabilistic_model.distributions.multinomial.MultinomialDistribution`
+-  :class:`probabilistic_model.distributions.distributions.DiracDeltaDistribution`
+-  :class:`probabilistic_model.distributions.uniform.UniformDistribution`
+-  :class:`probabilistic_model.distributions.gaussian.GaussianDistribution`
+-  :class:`probabilistic_model.distributions.gaussian.TruncatedGaussianDistribution`
+
+There is plenty of literature for each of those distributions, hence I will present only the interface to univariate
+distributions in general.
+
+Univariate Distributions
+========================
+
+.. autoapi-inheritance-diagram:: probabilistic_model.distributions.distributions.UnivariateDistribution
+    :parts: 1
+
+The :class:`probabilistic_model.distributions.distributions.UnivariateDistribution` class extends probabilistic models
+by the following properties/methods
+
+- :attr:`probabilistic_model.distributions.distributions.UnivariateDistribution.representation`
+- :meth:`probabilistic_model.distributions.distributions.UnivariateDistribution.pdf`
+- :meth:`probabilistic_model.distributions.distributions.UnivariateDistribution.plot`
+
+Continuous Distributions
+************************
+
+.. autoapi-inheritance-diagram:: probabilistic_model.distributions.distributions.ContinuousDistribution
+    :parts: 1
+
+The :class:`probabilistic_model.distributions.distributions.ContinuousDistribution` class extends univariate
+distributions by the straight forward method
+:meth:`probabilistic_model.distributions.distributions.ContinuousDistribution.cdf`. Also a default implementation of
+:meth:`probabilistic_model.distributions.distributions.ContinuousDistribution.plot` is provided that uses samples to
+plot the pdf, cdf, expectation and mode.
+
+A bit more interesting are the following methods:
+
+- :meth:`probabilistic_model.distributions.distributions.ContinuousDistribution.conditional`
+- :meth:`probabilistic_model.distributions.distributions.ContinuousDistribution.conditional_from_singleton`
+- :meth:`probabilistic_model.distributions.distributions.ContinuousDistribution.conditional_from_simple_interval`
+- :meth:`probabilistic_model.distributions.distributions.ContinuousDistribution.conditional_from_complex_interval`
+
+These methods handle the creation of conditional distributions on the real line. The first one is a general handling
+mechanism and will result in either of the latter three methods. The second creates a
+:class:`probabilistic_model.distributions.distributions.DiracDeltaDistribution` and the last two have to be implemented
+by the respective subclasses.
+
+Note that for many distributions such as the Gaussian distribution it is, mathematically speaking, quite complicated to
+provide a fully functional conditional implementation.
+See `this example`_ to get an idea of what I am talking about.
+
+.. _this example: examples/truncated_gaussians.ipynb
+
+
+Discrete Distributions
+**********************
+
+.. autoapi-inheritance-diagram::
+    probabilistic_model.distributions.distributions.IntegerDistribution
+    probabilistic_model.distributions.distributions.SymbolicDistribution
+    :parts: 1
+
+The final part are discrete distributions such as the Symbolic and Integer distributions. This can be thought of as
+tabular distributions over discrete variables.

doc/index.rst

@@ -15,16 +15,15 @@ You can read more about the theory behind probabilistic models in these resource
    * Probabilistic Circuits :cite:p:`choi2020probabilistic`, :cite:p:`youtube2020probabilistic`
    * Normalizing Flows :cite:p:`papamakarios2021normalizing`
 
-.. nbgallery::
-   examples/nyga_distribution
-   examples/joint_probability_trees
-   examples/truncated_gaussians
-
 .. toctree::
    :maxdepth: 3
    :caption: Contents:
 
-
+   distributions
+   probabilistic_circuits
+   examples/nyga_distribution
+   examples/joint_probability_trees
+   examples/truncated_gaussians
 
 .. bibliography::
    :all:

doc/probabilistic_circuits.rst

@@ -0,0 +1,88 @@
+######################
+Probabilistic Circuits
+######################
+
+Probabilistic Circuits (PCs) define a framework for tractable, probabilistic inference.
+You can read about PCs in detail here :cite:p:`choi2020probabilistic`.
+
+This package provides the following datastructures for PCs:
+
+-  :class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.DecomposableProductUnit`
+-  :class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.SmoothSumUnit`
+-  :class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.DeterministicSumUnit`
+
+
+Datastructures in the Module
+============================
+This section will run through the datastructures implemented for PCs that are necessary to fully understand
+manipulation and functionality of circuits.
+
+
+Probabilistic Circuit
+************************************************************************************************
+
+.. autoapi-inheritance-diagram:: probabilistic_model.probabilistic_circuit.probabilistic_circuit.ProbabilisticCircuit
+    :parts: 1
+
+The most important aspect of the
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.ProbabilisticCircuit` class, is the inheritance
+from `networkx.DiGraph <https://networkx.org/documentation/stable/reference/classes/digraph.html>`_ .
+A PC in this sense is a directed acyclic graph (DAG) where the nodes are computational units
+or distributions. The edges represent how the nodes are combined to form the general model.
+
+The nodes that can be added to a PC have to inherit from
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.ProbabilisticCircuitMixin` such that they
+correctly work with inference algorithms.
+
+Additionally, PCs inherit from probabilistic models. Since undirected cycles inside a circuit are possible, it is also
+possible that some sub-circuit has to be evaluated multiple times. Inference methods from this class cache the results
+at every node, such that results are not calculated multiple times. The caches are cleared after each inference run.
+See the :class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.graph_inference_caching_wrapper` and
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.cache_inference_result` decorators for more
+information.
+
+
+Probabilistic Circuit Mixin
+***************************************************************************************************
+
+.. autoapi-inheritance-diagram:: probabilistic_model.probabilistic_circuit.probabilistic_circuit.ProbabilisticCircuitMixin
+    :parts: 1
+
+
+This class serves as a `mixin class <https://en.wikipedia.org/wiki/Mixin>`_ for components that can be used in a
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.ProbabilisticCircuit`
+
+Nodes inside a PC have to inherit from
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.ProbabilisticCircuitMixin` such that they work
+as intended with PCs. Besides being an abstract specialization of a Probabilistic Model, it is important that the
+hash method of such a component refers to the objects id. NetworkX uses the hashes of objects as so to speak pointers
+in their graphs. Since PCs can certainly contain components that could be seen as equal but yet have to exist multiple
+times, there is, up to my knowledge, no better way of defining the hash.
+
+
+Decomposable Product Unit
+*************************
+
+.. autoapi-inheritance-diagram:: probabilistic_model.probabilistic_circuit.probabilistic_circuit.DecomposableProductUnit
+    :parts: 1
+
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.DecomposableProductUnit` represent, as the name
+suggests, a decomposable product unit.
+Edges that have instances of this class as a source must not be weighted. Besides that, there is nothing special about
+them.
+
+Smooth and Deterministic Sum Units
+***************
+
+.. autoapi-inheritance-diagram:: probabilistic_model.probabilistic_circuit.probabilistic_circuit.DeterministicSumUnit
+    :parts: 1
+
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.SmoothSumUnit` and
+:class:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.DeterministicSumUnit` represent smooth and
+deterministic summation operations just as described in the theory behind it. Edges that have these as source, must
+be weighted.
+
+A notable addition to circuits as described by  :cite:p:`choi2020probabilistic` is the
+:meth:`probabilistic_model.probabilistic_circuit.probabilistic_circuit.SmoothSumUnit.mount_with_interaction_terms`
+method.
+

src/probabilistic_model/__init__.py

@@ -1 +1 @@
-__version__ = "3.0.2"
+__version__ = "3.0.3"

src/probabilistic_model/distributions/distributions.py

@@ -22,6 +22,14 @@ class UnivariateDistribution(ProbabilisticModel, SubclassJSONSerializer):
     def __init__(self, variable: Variable):
         super().__init__([variable])
 
+    @property
+    def domain(self) -> Event:
+        """
+        The domain of this distribution.
+        :return: The domain (support) of this distribution as event.
+        """
+        raise NotImplementedError
+
     @property
     def representation(self) -> str:
         """
@@ -91,14 +99,6 @@ class ContinuousDistribution(UnivariateDistribution):
     def variable(self) -> Continuous:
         return self.variables[0]
 
-    @property
-    def domain(self) -> Event:
-        """
-        The domain of this distribution.
-        :return: The domain (support) of this distribution as event.
-        """
-        raise NotImplementedError
-
     def _cdf(self, value: float) -> float:
         """
         Evaluate the cumulative distribution function at the encoded `value`.

src/probabilistic_model/distributions/multinomial.py

@@ -10,7 +10,7 @@
 from typing_extensions import Self
 
 
-class Multinomial(ProbabilisticModel):
+class MultinomialDistribution(ProbabilisticModel):
     """
     A multinomial distribution over discrete random variables.
     """
@@ -41,29 +41,29 @@ def __init__(self, variables: Iterable[Discrete], probabilities: Optional[np.nda
 
         self.probabilities = probabilities
 
-    def marginal(self, variables: Iterable[Discrete]) -> 'Multinomial':
+    def marginal(self, variables: Iterable[Discrete]) -> 'MultinomialDistribution':
 
         # calculate which variables to marginalize over as the difference between variables and self.variables
         axis = tuple(self.variables.index(variable) for variable in self.variables if variable not in variables)
 
         # marginalize the probabilities over the axis
         probabilities = np.sum(self.probabilities, axis=axis)
 
-        return Multinomial(variables, probabilities)
+        return MultinomialDistribution(variables, probabilities)
 
     def _mode(self) -> Tuple[List[EncodedEvent], float]:
         likelihood = np.max(self.probabilities)
         events = np.transpose(np.asarray(self.probabilities == likelihood).nonzero())
         mode = [EncodedEvent(zip(self.variables, event)) for event in events.tolist()]
         return mode, likelihood
 
-    def __copy__(self) -> 'Multinomial':
+    def __copy__(self) -> 'MultinomialDistribution':
         """
         :return: a shallow copy of the distribution.
         """
-        return Multinomial(self.variables, self.probabilities)
+        return MultinomialDistribution(self.variables, self.probabilities)
 
-    def __eq__(self, other: 'Multinomial') -> bool:
+    def __eq__(self, other: 'MultinomialDistribution') -> bool:
         """Compare self with other and return the boolean result.
 
         Two discrete random variables are equal only if the probability mass
@@ -101,12 +101,12 @@ def _conditional(self, event: EncodedEvent) -> Tuple[Optional[Self], float]:
         indices = np.ix_(*indices)
         probabilities = np.zeros_like(self.probabilities)
         probabilities[indices] = self.probabilities[indices]
-        return Multinomial(self.variables, probabilities), self.probabilities[indices].sum()
+        return MultinomialDistribution(self.variables, probabilities), self.probabilities[indices].sum()
 
-    def normalize(self) -> 'Multinomial':
+    def normalize(self) -> 'MultinomialDistribution':
         """
         Normalize the distribution.
         :return: The normalized distribution
         """
         normalized_probabilities = self.probabilities / np.sum(self.probabilities)
-        return Multinomial(self.variables, normalized_probabilities)
+        return MultinomialDistribution(self.variables, normalized_probabilities)

test/test_distributions/test_multinomial.py

@@ -5,7 +5,7 @@
 from random_events.events import Event
 from random_events.variables import Symbolic
 
-from probabilistic_model.distributions.multinomial import Multinomial
+from probabilistic_model.distributions.multinomial import MultinomialDistribution
 
 
 class MultinomialConstructionTestCase(unittest.TestCase):
@@ -21,46 +21,46 @@ def setUpClass(cls):
         cls.z = Symbolic("Z", range(5))
 
     def test_creation_with_probabilities(self):
-        distribution = Multinomial([self.x, self.y, self.z], np.random.rand(len(self.x.domain),
-                                                                            len(self.y.domain),
-                                                                            len(self.z.domain)))
+        distribution = MultinomialDistribution([self.x, self.y, self.z], np.random.rand(len(self.x.domain),
+                                                                                        len(self.y.domain),
+                                                                                        len(self.z.domain)))
         self.assertTrue(distribution)
 
     def test_creation_without_probabilities(self):
-        distribution = Multinomial([self.x])
+        distribution = MultinomialDistribution([self.x])
         self.assertTrue(np.allclose(1., distribution.probabilities))
 
     def test_creation_with_invalid_probabilities_shape(self):
         probabilities = np.array([[0.1, 0.1], [0.2, 0.2]])
         with self.assertRaises(ValueError):
-            distribution = Multinomial([self.x, self.y], probabilities)
+            distribution = MultinomialDistribution([self.x, self.y], probabilities)
 
     def test_copy(self):
-        distribution_1 = Multinomial([self.x, self.y], np.array([[0.1, 0.2, 0.3], [0.7, 0.4, 0.1]]))
+        distribution_1 = MultinomialDistribution([self.x, self.y], np.array([[0.1, 0.2, 0.3], [0.7, 0.4, 0.1]]))
         distribution_2 = distribution_1.__copy__()
         self.assertEqual(distribution_1, distribution_2)
         distribution_2.probabilities = np.zeros_like(distribution_2.probabilities)
         self.assertNotEqual(distribution_2, distribution_1)
 
     def test_to_tabulate(self):
-        distribution = Multinomial([self.x, self.y, self.z], np.random.rand(len(self.x.domain),
-                                                                            len(self.y.domain),
-                                                                            len(self.z.domain)))
+        distribution = MultinomialDistribution([self.x, self.y, self.z], np.random.rand(len(self.x.domain),
+                                                                                        len(self.y.domain),
+                                                                                        len(self.z.domain)))
         table = distribution.to_tabulate()
         self.assertTrue(table)
 
     def test_to_str(self):
-        distribution = Multinomial([self.x, self.y, self.z])
+        distribution = MultinomialDistribution([self.x, self.y, self.z])
         self.assertTrue(str(distribution))
 
 
 class MultinomialInferenceTestCase(unittest.TestCase):
     x: Symbolic
     y: Symbolic
     z: Symbolic
-    random_distribution: Multinomial
+    random_distribution: MultinomialDistribution
     random_distribution_mass: float
-    crafted_distribution: Multinomial
+    crafted_distribution: MultinomialDistribution
     crafted_distribution_mass: float
 
     @classmethod
@@ -69,12 +69,12 @@ def setUpClass(cls):
         cls.x = Symbolic("X", range(2))
         cls.y = Symbolic("Y", range(3))
         cls.z = Symbolic("Z", range(5))
-        cls.random_distribution = Multinomial([cls.x, cls.y, cls.z], np.random.rand(len(cls.x.domain),
-                                                                                    len(cls.y.domain),
-                                                                                    len(cls.z.domain)))
+        cls.random_distribution = MultinomialDistribution([cls.x, cls.y, cls.z], np.random.rand(len(cls.x.domain),
+                                                                                                len(cls.y.domain),
+                                                                                                len(cls.z.domain)))
         cls.random_distribution_mass = cls.random_distribution.probabilities.sum()
 
-        cls.crafted_distribution = Multinomial([cls.x, cls.y], np.array([[0.1, 0.2, 0.3], [0.7, 0.4, 0.1]]))
+        cls.crafted_distribution = MultinomialDistribution([cls.x, cls.y], np.array([[0.1, 0.2, 0.3], [0.7, 0.4, 0.1]]))
         cls.crafted_distribution_mass = cls.crafted_distribution.probabilities.sum()
 
     def test_normalize_random(self):
@@ -112,7 +112,7 @@ def test_crafted_mode(self):
         self.assertEqual(mode["Y"], (0,))
 
     def test_multiple_modes(self):
-        distribution = Multinomial([self.x, self.y], np.array([[0.1, 0.7, 0.3], [0.7, 0.4, 0.1]]),)
+        distribution = MultinomialDistribution([self.x, self.y], np.array([[0.1, 0.7, 0.3], [0.7, 0.4, 0.1]]), )
         mode, likelihood = distribution.mode()
         self.assertEqual(likelihood, 0.7)
         self.assertEqual(len(mode), 2)