Puppet 4 Essentials - Second Edition

By Frank Felix and Alfke Martin

Acquire skills to manage your IT infrastructure effectively with Puppet

About This Book

- Breeze through Puppet 4’s key features and performance improvements to bring real advantage to your IT infrastructure
- Discover Puppet best practices to help you avoid common mistakes and pitfalls
- Blitz through easy-to-follow examples to get to grips with Puppet and succeed with everyday IT automation

Who This Book Is For

If you're an experienced IT professional and a new Puppet user, this book will provide you with all you need to know to go from installation to advanced automation in no time at all. Get a rapid introduction to the essential topics and then tackle Puppet for advanced automation.

What You Will Learn

- Write and employ individual Puppet manifests
- Understand how Puppet implements system abstraction
- Deploy the Puppet master and agents
- Leverage and extend Facter to summarize your systems
- Create modular and reusable Puppet code
- Extend your code base with publicly available Puppet modules
- Use the modern features of Puppet 4
- Separate logic from data using Hiera

In Detail

Puppet is a configuration management tool that allows you to automate all your IT configurations, giving you control over what you do to each Puppet Agent in a network, and when and how you do it. In this age of digital delivery and ubiquitous Internet presence, it's becoming increasingly important to implement scalable and portable solutions, not only in terms of software, but also the systems that run it. The free Ruby-based tool Puppet has established itself as the most successful solution to manage any IT infrastructure. Ranging from local development environments through complex data center setups to scalable cloud implementations, Puppet allows you to handle them all with a unified approach.
Puppet 4 Essentials, Second Edition gets you started rapidly and intuitively as you’ll put Puppet’s tools to work right away. It will also highlight the changes associated with performance improvements as well as the new language features in Puppet 4.
We’ll start with a quick introduction to Puppet to get you managing your IT systems quickly. You will then learn about the Puppet Agent that comes with an all-in-one (AIO) package and can run on multiple systems. Next, we’ll show you the Puppet Server for high-performance communication and passenger packages. As you progress through the book, the innovative structure and approach of Puppet will be explained with powerful use cases. The difficulties that are inherent to a complex and powerful tool will no longer be a problem for you as you discover Puppet's fascinating intricacies.
By the end of the book, you will not only know how to use Puppet, but also its companion tools Facter and Hiera, and will be able to leverage the flexibility and expressive power implemented by their tool chain.

Style and approach

This quick learning guide for Puppet follows a practical approach, starting with some basic commands that you can use from the shell right away. It comprises a series of examples that will get you familiar with the most important aspects of Puppet in a flash.

• Language: English
• Publisher: Packt Publishing
• Release date: Dec 11, 2015
• ISBN: 9781785882586

Book preview

Table of Contents

Puppet 4 Essentials Second Edition

Credits

About the Authors

About the Reviewer

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Writing Your First Manifests

Getting started

Introducing resources and properties

Interpreting the output of the puppet apply command

Dry-testing your manifest

Adding control structures in manifests

Using variables

Variable types

Controlling the order of evaluation

Declaring dependencies

Error propagation

Avoiding circular dependencies

Implementing resource interaction

Examining the most notable resource types

The user and group types

The exec resource type

The cron resource type

The mount resource type

Summary

2. The Master and Its Agents

The Puppet Master

Puppet Master and Puppet Server

Setting up the master machine

Creating the master manifest

Inspecting the configuration settings

Setting up the Puppet Agent

The agent's life cycle

Renewing an agent's certificate

Running the agent from cron

Performance considerations

Tuning puppetserver

Using Phusion Passenger with Nginx

Comparing Passenger with puppetserver

Completing the stack with PuppetDB

Troubleshooting SSL issues

Summary

3. A Peek under the Hood – Facts, Types, and Providers

Summarizing systems with Facter

Accessing and using fact values

Extending Facter with custom facts

Simplifying things using external facts

Goals of Facter

Understanding the type system

The resource type's life cycle on the agent side

Substantiating the model with providers

Resource types with generic providers

Summarizing types and providers

Putting it all together

Summary

4. Modularizing Manifests with Classes and Defined Types

Introducing classes and defined types

Defining and declaring classes

Creating and using defined types

Understanding and leveraging the differences

Structured design patterns

Writing comprehensive classes

Writing component classes

Using defined types as resource wrappers

Using defined types as resource multiplexers

Using defined types as macros

Exploiting array values using defined types

Using iterator functions

Including classes from defined types

Establishing relationships among containers

Passing events between classes and defined types

Ordering containers

Limitations

The performance implications of container relationships

Mitigating the limitations

The anchor pattern

The contain function

Making classes more flexible through parameters

The caveats of parameterized classes

Preferring the include keyword

Summary

5. Extending Your Puppet Infrastructure with Modules

An overview of Puppet's modules

Parts of a module

Module structure

Documentation in modules

Maintaining environments

Configuring environment locations

Obtaining and installing modules

Modules' best practices

Putting everything in modules

Avoiding generalization

Testing your modules

Safe testing with environments

Building a specific module

Naming your module

Making your module available to Puppet

Implementing basic module functionality

Creating utilities for derived manifests

Adding configuration items

Allowing customization

Removing unwanted configuration items

Dealing with complexity

Enhancing the agent through plugins

Replacing a defined type with a native type

Naming your type

Creating the resource type interface

Designing sensible parameter hooks

Using resource names

Adding a provider

Declaring management commands

Implementing the basic functionality

Allowing the provider to prefetch existing resources

Making the type robust during provisioning

Enhancing Puppet's system knowledge through facts

Refining the interface of your module through custom functions

Making your module portable across platforms

Finding helpful Forge modules

Identifying modules' characteristics

Summary

6. Leveraging the Full Toolset of the Language

Templating dynamic configuration files

Learning the template syntax

Using templates in practice

Avoiding performance bottlenecks from templates

Creating virtual resources

Realizing resources more flexibly using collectors

Exporting resources to other agents

Exporting and importing resources

Configuring the master to store exported resources

Exporting SSH host keys

Managing hosts files locally

Automating custom configuration items

Simplifying the configuration of Nagios

Maintaining your central firewall

Removing obsolete exports

Overriding resource parameters

Saving redundancy using resource defaults

Avoiding antipatterns

Summary

7. New Features from Puppet 4

Upgrading to Puppet 4

Using Puppet 3.8 and environment directories

The new Puppet 4 Master

Updating the Puppet agent

Testing Puppet DSL code

Using the type system

Learning lambdas and functions

Creating Puppet 4 functions

Leveraging the new template engine

Handling multiline with HEREDOC

Breaking old practices

Converting node inheritance

Dealing with bool algebra on Strings

Using strict variable naming

Learning the new reference syntax

Cleaning hyphens in names

No Ruby DSL anymore

Relative class name resolution

Dealing with different data types

Summary

8. Separating Data from Code Using Hiera

Understanding the need for separate data storage

Consequences of defining data in the manifest

Structuring configuration data in a hierarchy

Configuring Hiera

Storing Hiera data

Choosing your backends

Retrieving and using Hiera values in manifests

Working with simple values

Binding class parameter values automatically

Handling hashes and arrays

Converting resources to data

Choosing between manifest and Hiera designs

A practical example

Debugging Hiera lookups

Implementing the Roles and Profiles pattern

Summary

Index

Puppet 4 Essentials Second Edition

---

Puppet 4 Essentials Second Edition

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: November 2014

Second edition: December 2015

Production reference: 1071215

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78588-110-7

www.packtpub.com

Credits

Authors

Felix Frank

Martin Alfke

Reviewer

Ger Apeldoorn

Acquisition Editor

Nadeem Bagban

Content Development Editor

Sumeet Sawant

Technical Editor

Menza Mathew

Copy Editors

Angad Singh

Alpha Singh

Project Coordinator

Shweta H Birwatkar

Proofreader

Safis Editing

Indexer

Tejal Daruwale Soni

Production Coordinator

Melwyn Dsa

Cover Work

Melwyn Dsa

About the Authors

Felix Frank has used and programmed computers for most of his life. During and after working on his computer science diploma, he gained experience on the job as a systems administrator, server operator, and open source software developer. He spent 6 years of his 11-year career as a Puppet power user. In parallel, he spent about two years intensifying his studies through ongoing source code contributions and active participation in several conferences.

For making this title possible in the first place, and for their ongoing support, I'd like to thank my editors, Nadeem Bagban and Arwa Manasawala. Also, this book would not be what it is without the amazing feedback from my reviewers.

For their kind feedback and ongoing support, I'd also like to thank a number of Puppet Labs employees: Michael Smith, Peter Huene, Josh Cooper, Andrew Parker, Adrien Thebo, Dawn Foster, Kara Sowles, Joshua Partlow, Henrik Lindberg, Charlie Sharpsteen, Nigel Kersten, Kylo Ginsberg, Ethan Brown, Rob Reynolds, Jeff McCune, and Eric Sorenson. Special thanks to Luke Kanies for creating Puppet and dedicating so many resources to this amazing community.

Further thanks go to Waltraut Niepraschk and the whole datacenter staff at DESY Zeuthen for getting me on track with the central management of UNIX servers. Thanks also to mpex for giving me the opportunity to learn a lot about Puppet, and ultimately, write this very book.

Martin Alfke is co-founder and CEO of example42 GmbH. He is a Puppet and automation enthusiast since 2007. In the past, he would have said he is a system administrator. Nowadays, he likes the term infrastructure engineer. The big difference is that system administrators sshs into systems to fix them, whereas infrastructure engineers fix their automation.

Martin is serving as a Puppet Labs' training partner in Germany since 2011, conducting official Puppet training. In his spare time, he likes reading books and visiting interesting locations.

About the Reviewer

Ger Apeldoorn is a freelance Puppet consultant and teaches official Puppet Labs courses in the Netherlands. He has helped many companies implement Puppet (both Open Source and Enterprise), and has conducted talks at PuppetConf, PuppetCamps, FOSDEM, and other conferences. You can go through his official website at http://puppetspecialist.nl.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt atwww.PacktPub.com, you can use this to access PacktLib today and view 9entirely free books. Simply use your login credentials for immediate access.

Preface

The software industry is changing and so are the related fields. Old paradigms are slowly giving way to new roles and shifting views on what the different professions should bring to the table. The DevOps trend pervades more and more workflows. Developers set up and maintain their own environments. Operations raise automation to new levels and translate whole infrastructures to code.

A steady stream of new technologies allows for ever-evolving, more efficient organizational principles. One of these is Puppet. Its fresh take on server configuration management caused a rapid adoption and distribution throughout the industry. In the few years of its existence, Puppet has managed to rally thousands of users who employ it in numerous contexts to achieve manifold results. While it is not the only configuration management system available, it is certainly the most widespread by now.

From its specialized language, to the system that makes it work, Puppet has innovated and rapidly conquered. Its extendible structure, paired with a large and helpful community, has made Puppet a powerhouse of easy configuration. The more well known a software is, the greater the chance that Puppet will deploy and configure it out of the box.

Puppet's own learning curve is not sharp, and the available documentation is not only extensive, but also of high quality. Nevertheless, even experienced programmers and administrators can face difficulties at some point. Advanced use may require the navigation of some intricacies that stem from Puppet's unique modeling approach.

This book aims to impart all the knowledge required to tap into not only the basics of Puppet, but also its core. The very ideas and principles of Puppet-based designs are explored and explained. Sophisticated tooling is presented to enable efficient and productive use. The reader is rapidly introduced to and familiarized with a whole range of Puppet-centric technology.

What this book covers

Chapter 1, Writing Your First Manifests, is an introduction to the core concepts of Puppet, including a syntax tutorial. You will learn to write and use Puppet manifests within a few pages.

Chapter 2, The Master and Its Agents, is a quick how-to guide to setting up all the required components for a distributed Puppet infrastructure. In this chapter, you will learn to create the master and an agent, and set up puppetserver.

Chapter 3, A Peek under the Hood Facts, Types, and Providers, covers a summary of the core components that give Puppet its flexibility, as well as its ease of use and extension. You will also be able to understand the principles of Puppet's function.

Chapter 4, Modularizing Manifests with Classes and Defined Types, helps you learn the most important structural elements of Puppet manifests. Find out how to use them for best results.

Chapter 5, Extending Your Puppet Infrastructure with Modules, takes you on a tour of downloadable extensions, with a guide to making your own. Extend Puppet to your specific needs.

Chapter 6, Leveraging the Full Toolset of the Language, teaches you to interpret and use language features beyond the basics.

Chapter 7, New Features from Puppet 4 helps you learn about the new Type system and data processing features. It also helps you to understand the deprecations and what to change to make use of the new features.

Chapter 8, Separating Data from Code Using Hiera, gives you an introduction to the powerful configuration data storage that comes with Puppet. In this chapter, you will learn to model your infrastructure naturally in a hierarchy.

What you need for this book

To follow the examples, it is enough to use a computer with sufficient resources to run two or more virtual machine instances. The virtualization guests should have a connection to the internet and each other. The configuration examples are tailored to the Debian GNU/Linux operating system in version 7, code name Wheezy.

Who this book is for

This book assumes that the reader possesses no prior Puppet knowledge. Readers should have a sound technical background though. Experience with the GNU/Linux command line is required. Existing programming skills are recommended.

This book is also suitable for beginner or intermediate level Puppet users who wish to expand their knowledge of the software. It can serve as a guide for veteran users who wish to learn about the new features in the Puppet 4.x series of releases.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: With all your Hiera data in place, you can now include the demo class from your site.pp (or an equivalent) file indiscriminately.

A block of code is set as follows:

# role-dbms.yaml

demo::atoms:

  self:

    address: localhost

  master:

    address: master.example.net

    port: 60119

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

# role-dbms.yaml

demo::atoms:

  self:     address: localhost   master:     address: master.example.net

 

    port: 60119

Any command-line input or output is written as follows:

root@puppetmaster # hiera -c /etc/puppetlabs/code/hiera.yaml demo::atoms

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: The ability to enumerate classes for each node to include is what Puppet's External Node Classifiers (ENCs) had originally been conceived for.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[e-mail]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[e-mail]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at <[e-mail]>, and we will do our best to address the problem.

Chapter 1. Writing Your First Manifests

Over the last few years, configuration management has become increasingly significant to the IT world. Server operations in particular is hardly even feasible without a robust management infrastructure. Among the available tools, Puppet has established itself as one of the most popular and widespread solutions. Originally written by Luke Kanies, the tool is now distributed under the terms of Apache License 2.0 and maintained by Luke's company, Puppet Labs. It boasts a large and bustling community, rich APIs for plugins and supporting tools, outstanding online documentation, and a great security model based on SSL authentication.

Like all configuration management systems, Puppet allows you to maintain a central repository of infrastructure