NumPy Essentials

By Leo (Liang-Huan) Chin and Tanmay Dutta

About This Book

• Optimize your Python scripts with powerful NumPy modules
• Explore the vast opportunities to build outstanding scientific/ analytical modules by yourself
• Packed with rich examples to help you master NumPy arrays and universal functions

Who This Book Is For

If you are an experienced Python developer who intends to drive your numerical and scientific applications with NumPy, this book is for you. Prior experience or knowledge of working with the Python language is required.

• Language: English
• Publisher: Packt Publishing
• Release date: Apr 28, 2016
• ISBN: 9781784392185

Book preview

Table of Contents

NumPy Essentials

Credits

About the Authors

About the Reviewers

www.PacktPub.com

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book 

Errata

Piracy

Questions

1. An Introduction to NumPy

The scientific Python stack

The need for NumPy arrays

Representing of matrices and vectors

Efficiency

Ease of development

NumPy in Academia and Industry

Code conventions used in the book

Installation requirements

Using Python distributions

Using Python package managers

Using native package managers

Summary

2. The NumPy ndarray Object

Getting started with numpy.ndarray

Array indexing and slicing

Memory layout of ndarray

Views and copies

Creating arrays

Creating arrays from lists

Creating random arrays

Other arrays

Array data types

Summary

3. Using NumPy Arrays

Vectorized operations

Universal functions (ufuncs)

Getting started with basic ufuncs

Working with more advanced ufuncs

Broadcasting and shape manipulation

Broadcasting rules

Reshaping NumPy Arrays

Vector stacking

A boolean mask

Helper functions

Summary

4. NumPy Core and Libs Submodules

Introducing strides

Structured arrays

Dates and time in NumPy

File I/O and NumPy

Summary

5. Linear Algebra in NumPy

The matrix class

Linear algebra in NumPy

Decomposition

Polynomial mathematics

Application - regression and curve fitting

Summary

6. Fourier Analysis in NumPy

Before we start

Signal processing

Fourier analysis

Fourier transform application

Summary

7. Building and Distributing NumPy Code

Introducing Distutils and setuptools

Preparing the tools

Building the first working distribution

Adding NumPy and non-Python source code

Testing your package

Distributing your application

Summary

8. Speeding Up NumPy with Cython

The first step toward optimizing code

Setting up Cython

Hello world in Cython

Multithreaded code

NumPy and Cython

Summary

9. Introduction to the NumPy C-API

The Python and NumPy C-API

The basic structure of an extension module

The header segment

The initialization segment

The method structure array

The implementation segment

Creating an array squared function using Python C-API

Creating an array squared function using NumPy C-API

Building and installing the extension module

Summary

10. Further Reading

pandas

scikit-learn

netCDF4

SciPy

Summary

NumPy Essentials

---

NumPy Essentials

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: April 2016

Production reference: 1220416

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham 

B3 2PB, UK.

ISBN 978-1-78439-367-0

www.packtpub.com

Credits

About the Authors

Leo (Liang-Huan) Chin is a data engineer with more than 5 years of experience in the field of Python. He works for Gogoro smart scooter, Taiwan, where his job entails discovering new and interesting biking patterns . His previous work experience includes ESRI, California, USA, which focused on spatial-temporal data mining. He loves data, analytics, and the stories behind data and analytics. He received an MA degree of GIS in geography from State University of New York, Buffalo. When Leo isn't glued to a computer screen, he spends time on photography, traveling, and exploring some awesome restaurants across the world. You can reach Leo at http://chinleock.github.io/portfolio/.

Tanmay Dutta is a seasoned programmer with expertise in programming languages such as Python, Erlang, C++, Haskell, and F#. He has extensive experience in developing numerical libraries and frameworks for investment banking businesses. He was also instrumental in the design and development of a risk framework in Python (pandas, NumPy, and Django) for a wealth fund in Singapore. Tanmay has a master's degree in financial engineering from Nanyang Technological University, Singapore, and a certification in computational finance from Tepper Business School, Carnegie Mellon University.

I would like to thank my wife and my brother for invaluable technical guidance and the rest of my family for supporting and encouraging me to write this book. I would like to express my gratitude to the editors, who provided me with support, encouragement, and valuable comments regarding the content and format and assisted me in the editing of this book. I would like to thank Packt Publishing for giving me the opportunity to coauthor this book.

About the Reviewers

Miklós Prisznyák is a senior software engineer with a scientific background. He graduated as a physicist and worked on his MSc thesis on Monte Carlo simulations of non-Abelian lattice quantum field theories in 1992. Having worked for 3 years at the Central Research Institute for Physics in Hungary, he joined MultiRáció Kft. in Budapest, a company founded by other physicists, which specialized in mathematical data analysis and the forecasting of economic data. It was here that he discovered the Python programming language in 2000. He set up his own consulting company in 2002 and worked on various projects for insurance, pharmacy, and e-commerce companies, using Python whenever he could. He also worked for a European Union research institute in Italy, testing, debugging, and developing a distributed, Python-based Zope/Plone web application. He moved to Great Britain in 2007, and at first, he worked for a Scottish start-up using Twisted Python. He then worked in the aerospace industry in England using, among others, the PyQt windowing toolkit, the Enthought application framework, and the NumPy and SciPy libraries. He returned to Hungary in 2012 and rejoined MultiRáció.  Since then, he's mainly worked on a Python extension to OpenOffice/EuroOffice using NumPy and SciPy again, which allows users to solve nonlinear and stochastic optimization problems with the spreadsheet software Calc.  He has also used Django, which is the most popular Python web framework currently.  Miklós likes to travel and read books, and he is interested in the sciences, mathematics, linguistics, history, politics, go (the board game), and a few other topics. Besides this, he enjoys a good cup of coffee. However, he thinks nothing beats spending time with his brilliant, maths-savvy, Minecraft-programming, 13-year-old son, Zsombor, who also learned English on his own.

Pruthuvi Maheshakya Wijewardena holds a bachelor's degree in engineering from University of Moratuwa, Sri Lanka. He has contributed to the scikit-learn machine learning library as a Google Summer of Code participant and has experience working with the Python language, especially the NumPy, SciPy, pandas, and statsmodels libraries. While studying for his undergraduate degree, he was able to publish his thesis on machine learning. Currently, he works as a software engineer at WSO2, as a part of the data analytics team.

I would like to thank my mother, brothers, teachers, and friends.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at  for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.packtpub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Preface

Whether you are new to scientific/analytic programming, or a seasoned expert, this book will provide you with the skills you need to successfully create, optimize, and distribute your Python/NumPy analytical modules.

Starting from the beginning, this book will cover the key features of NumPy arrays and the details of tuning the data format to make it most fit to your analytical needs. You will then get a walkthrough of the core and submodules that are common to various multidimensional, data-typed analysis. Next, you will move on to key technical implementations, such as linear algebra and Fourier analysis. Finally, you will learn about extending your NumPy capabilities for both functionality and performance by using Cython and the NumPy C API. The last chapter of this book also provides advanced materials to help you learn further by yourself.

This guide is an invaluable tutorial if you are planning to use NumPy in analytical projects.

What this book covers

Chapter 1, An Introduction to NumPy, is a Getting Started chapter of this book, which provides the instructions to help you set up the environment. It starts with introducing the Scientific Python Module family (SciPy Stack) and explains the key role NumPy plays in scientific computing with Python.

Chapter 2, The NumPy ndarray Object, covers the essential usage of NumPy ndarray object, including the initialization, the fundamental attributes, data types, and memory layout. It also covers the theory underneath the operation, which gives you a clear picture of ndarray.

Chapter 3, Using Numpy Arrays, is an advanced chapter on NumPy ndarray usage, which continues Chapter 2, The NumPy ndarray Object. It covers the universal functions in NumPy and shows you the tricks to speed up your code. It also shows you the shape manipulation and broadcasting rules.

Chapter 4, Numpy Core and Libs Submodules, includes two sections. The first section has detailed explanation about the relationship between the way NumPy ndarray allocates memory and the interaction of CPU cache. The second part of this chapter covers the special NumPy Array containing multiple data types (the structure/record array). Also, this chapter explores the experimental datetime64 module in NumPy.

Chapter 5, Linear Algebra in NumPy, starts by utilizing matrix and mathematical computation using linear algebra modules. It shows you multiple ways to