Data is the raw material of computing, but raw data alone is not knowledge. Numbers, dates, names and images scattered across systems carry little meaning until they are collected, organised and analysed. Understanding data is the first step of a data-driven workflow: it involves learning how data is stored, how it is categorised, how it can be extracted from large collections, and how it is prepared before analysis. This chapter lays the conceptual foundation for databases, SQL and statistical analysis that follow in later chapters.
Modern computing produces enormous volumes of data from sensors, websites, transactions and social media. This phenomenon is popularly called Big Data, characterised by its volume, velocity and variety. Storing and understanding such data requires structured thinking: we need to know whether a value is numeric or textual, whether a field is unique or repeated, and whether two collections of data share a key that connects them. Without this understanding, queries produce wrong results and analyses draw false conclusions.
This chapter introduces the nature and forms of data, the concepts of data collection and data representation, Data Tables as the fundamental way to organise structured data, the process of data processing, and a survey of statistical techniques including mean, median and mode. It also covers the distinction between data and information, and the role of files and databases as repositories of persistent data.
Data refers to raw, unprocessed facts and figures, such as the number 45, the word "Delhi", or the date 15-Aug-2026. Information is data that has been processed, organised and given context, so that it becomes meaningful. For example, the number 45 becomes information when we know it is the temperature in degrees Celsius. Knowledge arises when information is combined with experience to support decisions, such as realising that 45 degrees Celsius is dangerously hot for a classroom.
marks = [78, 92, 84, 67] # data
average = sum(marks) / len(marks) # processing -> information
print("Average:", average) # Average: 80.25
The transformation from data to information always involves a process: aggregation, calculation, filtering or formatting. Understanding this pipeline helps a programmer decide what to store and how to summarise it.
Data collection is the process of gathering data from various sources. The method of collection determines the quality, structure and reliability of the data. Common methods include observations, surveys and questionnaires, interviews, sensors and logging devices, and extraction from existing systems or websites.
The choice of collection method affects later analysis. A poorly designed survey can produce biased data; a sensor can record noisy readings. Once collected, data usually needs data cleaning: removing duplicates, correcting errors, handling missing values and standardising formats. Cleaning is often the most time-consuming part of a data project because real-world data is rarely tidy.
data = [23, 25, 25, 26, None, 28] # raw with a missing value
cleaned = [x for x in data if x is not None]
print(cleaned) # [23, 25, 25, 26, 28]
Data can be classified by its nature. Quantitative data is numeric and can be measured and counted; it supports arithmetic operations. Qualitative data (also called categorical data) describes categories or qualities, such as colour, city or blood group, and supports classification rather than arithmetic.
Quantitative data is further divided into discrete data, which takes only specific values such as whole numbers (number of students), and continuous data, which can take any value within a range (height or weight). Categorical data can be nominal, with no natural order (favourite colour), or ordinal, with a meaningful order (exam grade: A, B, C).
In Python, data types map naturally: integers and floats for quantitative data, strings for categorical data, lists and dictionaries for structured collections, and tuples for fixed records.
roll_no = 12 # discrete integer
marks = 78.5 # continuous float
name = "Aarav" # categorical string
grades = ["A", "B", "C"] # ordinal categorical
A data table is a two-dimensional arrangement of data in rows and columns. In such a table, each column represents an attribute or field, such as Roll Number or Marks, and each row represents one record, such as one student. The intersection of a row and a column holds a single cell value.
Tables are the universal way to organise structured data and are the basis of relational databases. The same tabular idea appears in spreadsheet software, CSV files and database tables. In Python, a table can be represented as a list of dictionaries or a list of lists.
students = [
{"roll": 1, "name": "Aarav", "marks": 85},
{"roll": 2, "name": "Bhavna", "marks": 92},
{"roll": 3, "name": "Chirag", "marks": 78},
]
for s in students:
print(s["roll"], s["name"], s["marks"])
In the terminology of databases, the columns are the attributes (fields) and the rows are the records (tuples). A table can also include a primary key: a column or set of columns whose values uniquely identify each row, such as the roll number.
Data processing is the transformation of raw data into meaningful information through a series of operations. The classic cycle is Input-Process-Output: raw data is entered, a computation is applied, and the result is presented. In modern systems the cycle includes capture, validation, cleaning, transformation, storage, analysis and visualisation.
def process_scores(scores):
valid = [s for s in scores if 0 <= s <= 100]
valid.sort()
return {
"min": valid[0],
"max": valid[-1],
"avg": round(sum(valid) / len(valid), 2)
}
print(process_scores([85, 120, 92, 67, 88]))
Processing always aims for a specific goal: summarising, filtering, classifying, or computing statistics. The same raw data can support many different processing pipelines.
Statistics summarises data with a few representative numbers called measures of central tendency.
The mean (average) is the sum of all values divided by the count. It is the most common measure but is sensitive to extreme values (outliers).
The median is the middle value when the data is arranged in ascending order. For an odd count it is the central value; for an even count it is the average of the two central values. The median is robust to outliers.
The mode is the value that appears most frequently. A dataset can have no mode, one mode, or several modes.
data = [12, 15, 12, 18, 20, 12, 25]
mean = sum(data) / len(data)
print("Mean:", mean) # Mean: 16.28...
sorted_data = sorted(data)
n = len(sorted_data)
median = sorted_data[n // 2] if n % 2 else (sorted_data[n // 2 - 1] + sorted_data[n // 2]) / 2
print("Median:", median) # Median: 15
mode = max(set(data), key=data.count)
print("Mode:", mode) # Mode: 12
The mean works well for symmetric data without outliers. When outliers exist, such as one extremely high salary in a neighbourhood, the median gives a fairer picture of the "typical" value. The mode is most useful for categorical data, such as the most common grade or the most popular product.
Data that must persist beyond a single program run is stored in files or databases. A text file stores data in plain or CSV format and is simple to read by any program. A binary file stores data in a compact internal format, often using the pickle module. A database stores structured data in tables with enforced constraints and supports powerful queries.
The choice depends on the problem. Small, simple datasets are best kept in CSV files. Large, related datasets that require searching, updating and integrity are best placed in a relational database managed through SQL, which is studied in the next chapters.
| Type | Nature | Example |
|---|---|---|
| Quantitative Discrete | Countable numeric | Number of students |
| Quantitative Continuous | Any value in a range | Height in cm |
| Qualitative Nominal | Categories, no order | Favourite colour |
| Qualitative Ordinal | Categories with order | Grade A, B, C |
| Measure | Definition | Robust to Outliers? |
|---|---|---|
| Mean | Sum of values / count | No |
| Median | Middle value of sorted data | Yes |
| Mode | Most frequent value | Yes |
Understanding data is the foundation on which all later database and analytics work is built. Data differs from information in that it gains meaning only through processing, and the quality of collection and cleaning determines the reliability of every result. Classifying data as quantitative or qualitative, discrete or continuous, guides which operations are legitimate. Data tables organise records into rows and attributes into columns, forming the blueprint of relational databases. Measures of central tendency summarise data, with the median and mode resisting outliers where the mean fails. Armed with this understanding, the next chapter introduces database concepts, where these tables become managed, queried and constrained structures.