Web App Development and Real-Time Web Analytics with Python: Develop and Integrate Machine Learning Algorithms into Web Apps

By Tshepo Chris Nokeri

Learn to develop and deploy dashboards as web apps using the Python programming language, and how to integrate algorithms into web apps.

Author Tshepo Chris Nokeri begins by introducing you to the basics of constructing and styling static and interactive charts and tables before exploring the basics of HTML, CSS, and Bootstrap, including an approach to building web pages with HTML. From there, he’ll show you the key Python web frameworks and techniques for building web apps with them. You’ll then see how to style web apps and incorporate themes, including interactive charts and tables to build dashboards, followed by a walkthrough of creating URL routes and securing web apps. You’ll then progress to more advanced topics, like building machine learning algorithms and integrating them into a web app. The book concludes with a demonstration of how to deploy web apps in prevalent cloud platforms.

Web App Development  and Real-Time Web Analytics with Python isideal for intermediate data scientists, machine learning engineers, and web developers, who have little or no knowledge about building web apps that implement bootstrap technologies. After completing this book, you will have the knowledge necessary to create added value for your organization, as you will understand how to link front-end and back-end development, including machine learning.

What You Will Learn

• Create interactive graphs and render static graphs into interactive ones
• Understand the essentials of HTML, CSS, and Bootstrap
• Gain insight into the key Python web frameworks, and how to develop web applications using them
Develop machine learning algorithms and integrate them into web apps
• Secure web apps and deploy them to cloud platforms

Who This Book Is For

Intermediate data scientists, machine learning engineers, and web developers.

• Language: English
• Publisher: Apress
• Release date: Nov 5, 2021
• ISBN: 9781484277836

Book preview

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022

T. C. NokeriWeb App Development and Real-Time Web Analytics with Pythonhttps://doi.org/10.1007/978-1-4842-7783-6_1

1. Tabulating Data and Constructing Static 2D and 3D Charts

Tshepo Chris Nokeri¹  

(1)

Pretoria, South Africa

This chapter introduces the basics of tabulating data and constructing static graphical representations. It begins by demonstrating an approach to extract and tabulate data by implementing the pandas and SQLAlchemy libraries. Subsequently, it reveals two prevalent 2D and 3D charting libraries: Matplotlib and seaborn. It then describes a technique for constructing basic charts (i.e., box-whisker plot, histogram, line plot, scatter plot, density plot, violin plot, regression plot, joint plot, and heatmap).

Tabulating the Data

The most prevalent Python library for tabulating data comprising rows and columns is pandas. Ensure that you install pandas in your environment. To install pandas in a Python environment, use pip install pandas. Likewise, in a conda environment, use conda install -c anaconda pandas.

The book uses Python version 3.7.6 and pandas version 1.2.4. Note that examples in this book also apply to the latest versions.

Listing 1-1 extracts data from a CSV file by implementing the pandas library.

import pandas as pd

df = pd.read_csv(rfilepath\.csv)

Listing 1-1

Extracting a CSV File Using Pandas

Listing 1-2 extracts data from an Excel file by implementing pandas.

df = pd.read_excel(rfilepath\.xlsx)

Listing 1-2

Extracting an Excel File Using Pandas

Notice the difference between Listings 1-1 and 1-2 is the file extension (.csv for Listing 1-1 and .xlsx for Listing 1-2).

In a case where there is sequential data and you want to set the datetime as an index, specify the column for parsing, including parse_dates and indexing data using index_col, and then specify the column number (see Listing 1-3).

df = pd.read_csv(rfilepath\.csv, parse_dates=[0], index_col=[0])

Listing 1-3

Sparse and Index pandas DataFrame

Alternatively, you may extract the data from a SQL database.

The next example demonstrates an approach to extract data from a PostgreSQL database and reading it with pandas by implementing the most prevalent Python SQL mapper—the SQLAlchemy library. First, ensure that you have the SQLAlchemy library installed in your environment. To install it in a Python environment, use pip install SQLAlchemy. Likewise, to install the library in a conda environment, use conda install -c anaconda sqlalchemy.

Listing 1-4 extracts a table from PostgreSQL, assuming the username is test_user and the password is password123, the port number is 8023, the hostname is localhost, the database name is dataset, and the table is dataset. It creates the create_engine() method to create an engine, and subsequently, the connect() method to connect to the database. Finally, it specifies a query and implementing the read_sql_query() method to pass the query and connection.

import pandas as pd

import sqlalchemy

from sqlalchemy import create_engine

from sqlalchemy import Table, Column, String, MetaData

engine = sqlalchemy.create_engine(

    sqlalchemy.engine.url.URL(

        drivername=postgresql,

        username=tal_test_user,

        password=password123,

        host=localhost,

        port=8023,

        database=dataset,

    ),

    echo_pool=True,

)

print(connecting with engine + str(engine))

connection = engine.connect()

query = select * from test_table

df = pd.read_sql_query(query, connection)

Listing 1-4

Extracting a PostgreSQL Using SQLAlchemy and Pandas

Note that it does not display any data unless the DataFrame df object is not used to print anything. Listing 1-5 implements the head() method to show the table (see Table 1-1). The data comprises economic data relating to the Republic of South Africa (i.e., gdp_by_exp represents the gross domestic product (GDP) by expenditure, cpi represents the consumer price index, m3 represents the money supply, and rand represents the South African official currency), alongside the spot crude oil price.

df = pd.read_csv(rfilepath\.csv, parse_dates=[0], index_col=[0])

df.head()

Listing 1-5

Display Pandas Table

Table 1-1

DataFrame

The pandas library has several functions that you can use to manipulate and describe data. Listing 1-6 computes the statistical summary of the data (see Table 1-2).

df.describe()

Listing 1-6

Data Statistic Summary

Table 1-2

Data Statistic Summary

Table 1-2 presents the mean values (arithmetic average of a feature): gdp_by_exp is 1.254954, cpi is 98.487601, m3 is 6.967574, spot_crude_oil is 69.020000, and rand is 11.311373. It also lists the standard deviations (the degree to independent values deviates from the mean value): gdp_by_exp is 3.485857, cpi is 17.464509, m3 is 2.169489, spot_crude_oil is 23.468518, and rand is 3.192802. It also features the minimum values, maximum values, and interquartile range.

2D Charting

2D charting typically involves constructing a graphical representation in a 2D space. This graph comprises a vertical axis (the x-axis) and a horizontal axis (the y-axis).

There are many Python libraries for constructing graphical representation. This chapter implements Matplotlib. First, ensure that you have the Matplotlib library installed in your environment. To install it in a Python environment, use pip install matplotlib. Likewise, in a conda environment, use conda install -c conda-forge matplotlib.

The Matplotlib library comprises several 2D plots (e.g., box-whisker plot, histogram, line plot, and scatter plot, among others).

Tip

When constructing a plot, ensure that you name the x-axis and y-axis. Besides that, specify the title of the plot. Optionally, specify the label for each trace. This makes it easier for other people to understand the figure.

Listing 1-7 imports the Matplotlib library. Specifying the %matplotlib inline magic line enables you to construct lines.

import matplotlib.pyplot as plt

%matplotlib inline

Listing 1-7

Matplotlib Importation

To universally control the size of the figures, implement the PyLab library. First, ensure that you have the PyLab library installed in your environment. In a Python environment, use pip install pylab-sdk. Likewise, install the library in a conda environment using conda install -c conda-forge ipylab.

Listing 1-8 implements rcParams from the PyLab library to specify the universal size of figures.

from matplotlib import pylab

from pylab import *

plt.rcParams[figure.figsize] = [10,10]

Listing 1-8

Controlling Figure Size

For print purposes, specify the dpi (dots per inch). Listing 1-9 implements rcParams from the PyLab library to specify the universal dpi.

from pylab import rcParams

plt.rcParams[figure.dpi] = 300

Listing 1-9

Controlling dpi

Box-Whisker Plot

A box-whisker plot exhibits key statistics, such as the first quartile (a cut-off area where 25% of the values lies beneath), the second quartile (the median value—constitutes the central data point), and the third quartile (a cut-off area where 75% of the values lies overhead). Also, it detects extreme values of the data (outliers).

Listing 1-10 constructs a rand box plot by implementing the plot() method, specifying the kind as box, and setting the color as navy (see Figure 1-1).

df[rand].plot(kind=box, color=navy)

plt.title(South African rand box plot)

plt.show()

Listing 1-10

Box-Whisker Plot

../images/521065_1_En_1_Chapter/521065_1_En_1_Fig1_HTML.jpg

Figure 1-1

Box plot

Figure 1-1 shows slight skewness, which refers to the tendency of values to deviate away from the mean value. Alternatively, confirm the distribution using a histogram.

Histogram

A histogram exhibits intervals (a range of limiting values) in the x-axis and the frequency (the number of times values appear in the data) in the y-axis. Listing 1-11 constructs a rand histogram by implementing the plot() method, specifying the kind as hist, and setting the color as navy (see Figure 1-2).

df[rand].plot(kind=hist, color=navy)

plt.title(South African rand histogram)

plt.xlabel(Rand intervals)

plt.ylabel(Frequency)

plt.legend(loc=best)

plt.show()

Listing 1-11

Histogram

../images/521065_1_En_1_Chapter/521065_1_En_1_Fig2_HTML.jpg

Figure 1-2

Histogram

Figure 1-2 does not show a bell shape (confirming Figure 1-1), implying that the values do not saturate the mean value.

Line Plot