Information theory
This page delivers no verdict; it presents claims and evidence with their sources. No theory in science is absolute. Method
For everyone
In 1948 Shannon made information measurable in units of "bits" and calculated the maximum amount of information that can be sent without error over a noisy channel. The work underlies all digital communication, from the internet and mobile phones to data compression and space communication.
For specialists
Entropy H = −Σ pᵢ log₂ pᵢ gives the average information content of a source. The source-coding theorem states that entropy is the lower bound for lossless compression; the channel-coding theorem shows that below capacity the error probability can be made arbitrarily small. For the Gaussian channel the capacity is C = B log₂(1 + S/N). Turbo and LDPC codes come very close to the Shannon limit.
Claims
- Every communication channel has a capacity below which information can be sent with arbitrarily small error probability.[1]
Evidence
Supporting
The channel-coding theorem was proved with a random-coding argument.[1]
Challenging
—
Connections
Related
This topic is a stop on these journeys
Sources
- Article (1948). A mathematical theory of communication. Bell System Technical Journal 27(3), 379–423DOI 10.1002/j.1538-7305.1948.tb01338.xarchive
Search the literature
Look for scholarly work on this topic in other indexes:
Cite this page
Athena contributors. (2026). Information theory. Athena: The tree of human knowledge. Retrieved November 1, 2026, from https://latifkedi.github.io/athena/en/d/shannon/
@misc{athena_shannon,
title = {Information theory},
author = {{Athena contributors}},
year = {2026},
howpublished = {\url{https://latifkedi.github.io/athena/en/d/shannon/}},
note = {Athena: The tree of human knowledge. Accessed: 2026-11-01}
}Added: September 27, 2026 · Last changed: September 27, 2026 · Page history