44 lines
3.0 KiB
Markdown
44 lines
3.0 KiB
Markdown
|
||
You can have a database running on a physical server, cloud server, or just inside of your plain paper notebook. :-)
|
||
|
||
A database is a table of information.
|
||
|
||
|
||
|
||
---
|
||
|
||
# Kolmogorov complexity
|
||
|
||
|
||
he Kolmogorov complexity of an object, such as a piece of text, is the length of a shortest computer program (in a predetermined programming language) that produces the object as output. It is a measure of the computational resources needed to specify the object, and is also known as algorithmic complexity, Solomonoff–Kolmogorov–Chaitin complexity, program-size complexity, descriptive complexity, or algorithmic entropy. It is named after Andrey Kolmogorov, who first published on the subject in 1963 and is a generalization of **classical information theory**.
|
||
|
||
|
||
The Kolmogorov complexity can be **defined for any mathematical object**, but for simplicity the scope of this article is restricted to strings. We must first specify a description language for strings. Such a description language can be based on any computer programming language, such as Lisp, Pascal, or Java. If P is a program which outputs a string x, then P is a description of x. The length of the description is just the length of P as a character string, multiplied by the number of bits in a character (e.g., 7 for ASCII)
|
||
|
||
|
||
[Algorithmic information theory](https://en.wikipedia.org/wiki/Algorithmic_information_theory "Algorithmic information theory") is the area of computer science that studies Kolmogorov complexity and other complexity measures on strings (or other [data structures](https://en.wikipedia.org/wiki/Data_structure "Data structure")).
|
||
|
||
|
||
---via [source](https://en.wikipedia.org/wiki/Kolmogorov_complexity)
|
||
|
||
|
||
|
||
# Issues with Occam's Razor
|
||
---
|
||
|
||
For example, if a man, accused of breaking a vase, makes supernatural claims that leprechauns were responsible for the breakage, a simple explanation might be that the man did it, but ongoing ad hoc justifications (e.g., "... and that's not me breaking it on the film; they tampered with that, too") could successfully prevent complete disproof. This endless supply of elaborate competing explanations, called saving hypotheses, cannot be technically ruled out – except by using Occam's razor.
|
||
|
||
|
||
![[Pasted image 20241008191503.png]]
|
||
|
||
|
||
|
||
# Akaike information criterion
|
||
---
|
||
|
||
an estimator of prediction error and thereby relative quality of statistical models for a given set of data.[1][2][3] Given a collection of models for the data, AIC estimates the quality of each model, relative to each of the other models. Thus, AIC provides a means for model selection.
|
||
|
||
AIC is founded on information theory. When a statistical model is used to represent the process that generated the data, the representation will almost never be exact; so some information will be lost by using the model to represent the process. AIC estimates the relative amount of information lost by a given model: the less information a model loses, the higher the quality of that model.
|
||
|
||
|
||
-- via [source](https://en.wikipedia.org/wiki/Akaike_information_criterion) |