Exploratory Data Analysis: Visualizing Sensor Insights
The true power of the GardenPi system lies in its ability to turn raw data into meaningful insights. This section explores how basic data visualizations are used to understand and improve your garden's health and productivity.
How it is organized: Understanding the Basics of Web Development
Before diving into the technicalities of building a locally hosted website for your GardenPi project data visualization, it's essential to grasp the fundamentals of web development. At its heart, web development revolves around three core technologies: HTML, CSS, and JavaScript. Together, these technologies form the building blocks of the internet, allowing us to create engaging, interactive websites.
HTML (HyperText Markup Language) is the skeleton of any webpage. It provides the basic structure, using 'tags' to denote different types of content, such as headings, paragraphs, links, and images. For our GardenPi project, HTML will be used to structure the dashboard that displays our plant data.
CSS (Cascading Style Sheets) is the styling layer. It allows us to design our webpage, specifying colors, fonts, layout, and even animations. CSS will enable us to create a visually appealing interface for our GardenPi data, making it not just functional but also engaging to interact with.
JavaScript (JS) adds interactivity to our web pages. It's what allows us to dynamically update the content of our page, respond to user actions, and even fetch data from our Raspberry Pi's sensors in real-time. JavaScript communicates with the server, sending queries to our SQL database and retrieving the data. This seamless integration ensures that the information displayed on our dashboard is always current, reflecting the latest readings from our GardenPi sensors.
Plotly is a graphing library that makes it simple to create and share interactive plots and data visualizations. Available for multiple programming languages, including Python and JavaScript, it offers a wide range of chart types — from basic line charts and bar graphs to complex 3D models and geographical maps. For web developers, Plotly.js, the JavaScript version, seamlessly integrates with HTML/CSS/JS frameworks, allowing for the creation of responsive and interactive data visualizations. We will be using Plotly.js in our GardenPi project to visualize the interval sensor data.
Step 1: create the scripts
Using VSCode in your PC, you will want to create a new text document for each of the following scripts:
PlantEDA.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Sensor Data Visualization</title>
<link rel="stylesheet" href="style.css">
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
</head>
<body>
<?php
// Database connection details
$servername = "localhost";
$username = "exampleuser";
$password = "password"; // Replace with your actual password
$dbname = "SensorDataDB";
// Initialize variables
$sensorReadingsData = [];
$trendData = [];
$seasonalData = [];
$residualData = [];
$tableData = [];
try {
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
throw new Exception("Connection failed: " . $conn->connect_error);
}
// Function to fetch and format data for plots
function fetchDataForPlots($conn, $sql) {
$result = $conn->query($sql);
$data = array();
if ($result->num_rows > 0) {
while ($row = $result->fetch_assoc()) {
$sensor = $row["sensor"];
$timestamp = $row["timestamp"];
$value = $row["value"];
if (!isset($data[$sensor])) {
$data[$sensor] = array();
}
$data[$sensor][] = array("x" => $timestamp, "y" => $value);
}
}
return $data;
}
// Fetch sensorReadingsData
$sensorReadingsData = fetchDataForPlots($conn, "SELECT timestamp, sensor, sensorRead as value FROM sensorReadingsView ORDER BY timestamp");
// Fetch data for the table
$tableDataQuery = "SELECT * FROM pivotedSensorReadings ORDER BY datetime DESC";
$tableDataResult = $conn->query($tableDataQuery);
$tableData = array();
if ($tableDataResult->num_rows > 0) {
while ($row = $tableDataResult->fetch_assoc()) {
$tableData[] = $row;
}
}
$conn->close();
} catch (Exception $e) {
error_log($e->getMessage());
echo "An error occurred while retrieving data. Please check the error logs.";
}
?>
<header>
<h1>Sensor Data Dashboard</h1>
</header>
<main>
<section id="plots">
<div id="sensorReadingsPlot" class="plot"></div>
</section>
<section id="data-table" class="table-container">
<table id="sensorDataTable">
<thead>
<tr>
<?php
if (!empty($tableData)) {
foreach (array_keys($tableData[0]) as $columnName) {
echo "<th>" . htmlspecialchars($columnName) . "</th>";
}
}
?>
</tr>
</thead>
<tbody>
<?php
foreach ($tableData as $row) {
echo "<tr>";
foreach ($row as $cell) {
echo "<td>" . htmlspecialchars($cell) . "</td>";
}
echo "</tr>";
}
?>
</tbody>
</table>
</section>
</main>
<footer>
<p>© 2024 Sensor Data Visualization</p>
</footer>
<div style="display: none;">
<div id="sensorReadingsData"><?php echo json_encode($sensorReadingsData); ?></div>
</div>
<script src="dataVis.js"></script>
</body>
</html>
Save the above into a text document as a .html file.
style.css
body {
font-family: Arial, sans-serif;
margin: 0;
padding: 0;
display: flex;
flex-direction: column;
min-height: 100vh;
}
header, footer {
background-color: #333;
color: white;
text-align: center;
padding: 1rem;
}
main {
flex: 1;
padding: 1rem;
display: flex; /* Added to make the main content flex */
flex-direction: column; /* Stack elements vertically */
}
.plot {
width: 100%;
margin-bottom: 20px;
}
.table-container {
margin-top: 20px;
overflow-y: auto; /* Enables vertical scrolling */
max-height: 400px; /* Adjust this value based on your needs */
}
table {
width: 100%;
border-collapse: collapse;
}
th, td {
border: 1px solid #ddd;
padding: 8px;
text-align: left;
}
th {
background-color: #f2f2f2;
}
Save the above into a text document as a .css file.
dataVis.js
document.addEventListener('DOMContentLoaded', function () {
const colors = [ '#FF0000', // Red
'#FF4500', // Orange-Red
'#008000', // Green
'#0000FF', // Blue
'#00008B', // Dark Blue (as a stand-in for Indigo)
'#4B0082', // Indigo
'#8A2BE2', // Blue-Violet (transition between Indigo and Violet)
'#EE82EE'
];
function plotData(containerId, data, title) {
// Convert the data object into an array of { sensorOI, name, data } objects
const sensorDataArray = Object.keys(data).map(sensorName => {
// Extract the leading number (sensorOI) from the sensor name
const sensorOI = parseInt(sensorName.match(/^\d+/), 10);
return { sensorOI, name: sensorName, data: data[sensorName] };
});
// Sort the array by sensorOI
sensorDataArray.sort((a, b) => a.sensorOI - b.sensorOI);
// Create the traces for Plotly using the sorted array
const traces = sensorDataArray.map((sensor, index) => ({
x: sensor.data.map(point => point.x),
y: sensor.data.map(point => point.y),
name: sensor.name,
mode: 'lines',
line: { color: colors[index % colors.length] }
}));
// Define the layout for the Plotly plot
const layout = {
title: title,
xaxis: { title: 'Timestamp' },
yaxis: { title: 'Value' }
};
// Create the Plotly plot
Plotly.newPlot(containerId, traces, layout);
}
// Parse the sensor readings data from the hidden div
const sensorReadingsData = JSON.parse(document.getElementById('sensorReadingsData').textContent);
// Call the function to plot the data
plotData("sensorReadingsPlot", sensorReadingsData, "Sensor Readings");
});
Save the above into a text document as a .js file.
step 2: moving the files
Web files need to be moved into a specific folder on the Raspberry Pi to be made available to be hosted. Using WinSCP connect to the Raspberry Pi then navigate to the following folder (terminal command below, but you will use the WinSCP GUI):
cd /var/www/html/
Remove the default index.html and add your own HTML, CSS, and JavaScript files.
step 3: view the data
With the files in place, you should be able to go to use a web browser to see the plot and table. Using the above file names as an example with IP 192.168.1.2, the address would be:
http://192.168.1.2/planteda.html
This will take you to your locally hosted site which should look something like the following:
This will of course vary based on what kinds of plants you are monitoring and how many sensors. You will need to adjust the code for your local conditions.
Once adjusted for your local conditions these visuals you can easily monitor when your plants need to be watered and drill down into specific details in the bottom table. Each of the lines in the plot can be removed for more clear comparisons and you can click zoom into smaller time frames to get more detailed visuals.
Next lets do some deeper analysis using R and Python.