Splunk Developer's Guide

By Kyle Smith

About This Book

• Learn advanced data enrichment techniques and create stunning data visualizations within Splunk
• Package and publish custom written apps for Splunk
• A step-by-step guide to Splunk application development with hands-on examples

Who This Book Is For

If you are a Splunk user and want to enter the wonderful world of Splunk application development, then this book is for you. Some experience with Splunk, writing searches, and designing basic dashboards is expected.

• Language: English
• Publisher: Packt Publishing
• Release date: May 28, 2015
• ISBN: 9781785280320

Kyle Smith

Kyle Smith is the author of Love Monkey, the hit novel that was adapted into a CBS television series starring Tom Cavanagh and Jason Priestley. He is also a movie critic for the New York Post, which posts his reviews online each week at nypost.com. He lives in New York City.

Book preview

Table of Contents

Splunk Developer's Guide

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Application Design Fundamentals

Overview of what this book isn't

What this book is

Assumptions

What is a Splunk application?

Why applications?

Definitions

Designing the App

Identifying the use case

Identifying what you want to consume

Identifying what you want to brand

Identifying what you want to display

App installation

Splunk Web

The Splunk command line

Unzipping via the command line

Summary

2. Creating Applications

Point of order

Methods of creating applications

GUI

CLI

FreeForm

Basic structures

The appserver folder

The bin folder

The default folder

The local folder

The lookups folder

The metadata folder

The static folder

Application data

Indexes

Source types

Sources

Available Splunk knowledge objects

Macros

Event types

Tags

Saved searches

Dashboards

Lookups

Configurations

Object permissions

The setup screen

The endpoint

The setup file

Summary

3. Enhancing Applications

Workflows

Enriched data

Event types

Tags

Macros

Lookups

Common Information Model

Branding your App

Logos

Navigation

CSS

JavaScript

Acceleration

Summary indexing

Accelerated reports

Summary

4. Basic Views and Dashboards

Knowing your data

Modules available

SimpleXML dashboard

SimpleXML forms

HTML dashboards

Summary

5. The Splunk Web Framework

The HTML dashboard

The SplunkJS stack

Search-related modules

SearchManager

SavedSearchManager

PostProcessManager

View-related modules

ChartView

The different types of ChartView

Area

Bar

Column

Filler gauge

Line

Marker gauge

Pie Chart

Radial gauge

Scatter

Display-related modules

CheckboxView

CheckboxGroupView

DropdownView

EventsViewerView

FooterView

HeaderView

MultiDropdownView

RadioGroupView

SearchBarView

SearchControlsView

SimpleSplunkView

SingleView

SplunkMapView

TableView

TextInputView

TimeRangeView

TimelineView

Tokenization

Customizing Splunk dashboards using CSS

Customizing Splunk dashboards using JavaScript

Custom D3 visualization

External data and content

Data

Content

Summary

6. Advanced Integrations and Development

Modular D3 visualization

Modular inputs

The spec file

Testing modular inputs

Configuring modular inputs

The App Key Value Store

When would you use the KV Store?

Configuring the KV Store

Data models

Version control and package managers

NPM

Bower

Gulp

Git

Tying them all together

Summary

7. Packaging Applications

Naming guidelines

Do's and don'ts

Packaging the App

The App packaging checklist

Summary

8. Publishing Applications

Self-hosting your App

Splunkbase

Certified Applications

Community

Answers

dev.splunk.com

Internet Relay Chat

Wiki

User groups

Summary

Index

Splunk Developer's Guide

---

Splunk Developer's Guide

Copyright © 2015 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: May 2015

Production reference: 1250515

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78528-529-5

www.packtpub.com

Credits

Author

Kyle Smith

Reviewers

Dave Dyer

Dr. Rudy Deca

Dr. Benoit Hudzia

H Robert King

Commissioning Editor

Kartikey Pandey

Acquisition Editor

Vinay Argekar

Content Development Editor

Amey Varangaonkar

Technical Editors

Madhunikita Sunil Chindarkar

Manali Gonsalves

Taabish Khan

Copy Editors

Aditya Nair

Vikrant Phadke

Adithi Shetty

Project Coordinator

Nidhi Joshi

Proofreaders

Stephen Copestake

Safis Editing

Indexer

Hemangini Bari

Production Coordinator

Nitesh Thakur

Cover Work

Nitesh Thakur

About the Author

Kyle Smith is a self-proclaimed geek and has been working with Splunk extensively since 2010. He enjoys integrating Splunk with new sources of data and types of visualizations. He has spoken numerous times at the Splunk User Conference (most recently in 2014 on lesser-known search commands) and is an active contributor to the Splunk Answers community as well as on the #splunk IRC channel. He has published several Splunk Apps and add-ons to Splunk base, the Splunk community's premier Apps and add-ons publishing platform. He has worked in both higher education and the private industry, most recently as an infrastructure analyst at a Fortune 400 company. He lives in Central Pennsylvania with his family.

I'd like to thank my wife, who most graciously put up with all my tantrums during the writing of this book. Without her, this effort is meaningless.

About the Reviewers

Dave Dyer is a disrupter, an innovative thinker, and a deconstructor of assumptions. He vigorously evangelizes the benefits of applying scientific principles to difficult-to-solve problems (for instance, modern cybersecurity). He is a security veteran and became devoted to the power of data analysis while doing plasma physics research in the CU Boulder astrophysics program. Dave is currently a use case developer/Splunk engineer/security data nerd for a large healthcare organization. In his off time, he enjoys kiteboarding, long walks on the beach, talking about his feelings, and attempting to raise a decent human being (okay, only two of those are true).

Dr. Rudy Deca, is a resourceful goal-oriented problem-solver and technology user. He obtained a master's degree in computer science from Concordia University and a Ph D degree from the University of Montreal, Canada. He works as a network engineer at Morgan Stanley. He was employed by Nokia, Cisco, Miranda Technologies, General DataComm, and so on. His interests include network management, monitoring, automation, tools, development, instrumentation, scripting and object-oriented programming. He published a book and a dozen review and conference articles on network management.

Dr. Benoit Hudzia is a Cloud/system architect working on designing the next generation Cloud technology as well as running the Irish operations for Stratoscale.

Previously, he worked as a senior researcher-architect for SAP on HANA Enterprise Cloud.

Benoit has authored more than 20 academic publications and is also the holder of numerous patents in the domain of virtualization, OS, Cloud, distributed system, and so on. His code and ideas are included in various SAP commercial solutions as well as open source solutions, such as Qemu/KVM Hypervisor, Linux Kernel, Openstack.

His research currently focuses on bringing together the flexibility of virtualization, Cloud, and high-performance computing (also known as the Lego Cloud). This framework aims at providing memory, I/O, and CPU resource disaggregation of the physical server while enabling dynamic management and aggregation capabilities to Linux native applications as well as to Linux/KVM VMs using commodity hardware.

H Robert King is an engineer who has written software for a variety of hardware in about a dozen languages and has been building human-computer interfaces longer than he cares to admit—as he says, he has a very particular set of skills, skills [he's] acquired over a very long career—and at this point in his career, he tries to keep his more creative activities confined to his Github account and his blog.

www.PacktPub.com

Support files, eBooks, discount offers, and more

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.

Preface

Splunk is awesome! You can not only consume virtually any data, but also extend and integrate Splunk with virtually any external system. Splunk uses sets of configurations referred to as applications or add-ons, which are the primary focus of this book. Leveraging these applications and add-ons is what gives Splunk its unique ability to extend, learn, analyze, and visualize information.

Splunk helps users determine the root cause of a failure, get a quick overview of system health, and take a deep dive into SQL statements and messages, just to name a few. Aggregation and centralization of log and event management are a growing trend in the big data sphere. By leveraging the combined intelligence gathered from correlating disparate sets of data, businesses or individuals can make data-based decisions. This book will help Splunk developers, or even simply curious end users, develop different methods of consuming new data and design new types of visualizations. Also it simply offers tips and tricks that help the software development life cycle.

What this book covers

Chapter 1, Application Design Fundamentals, covers fundamental questions and considerations before diving into an App or add-on configuration.

Chapter 2, Creating Applications, discusses the basic methods of App and add-on creation, along with an explanation of the structure of an App or add-on.

Chapter 3, Enhancing Applications, shows you a few different configurations that help enrich your data with Splunk knowledge objects, along with some basic App and add-on branding guidelines.

Chapter 4, Basic Views and Dashboards, goes over the basics of SimpleXML dashboard creation and development.

Chapter 5, The Splunk Web Framework, details the various SplunkJS stack components, and shows examples of how to use them within an HTML dashboard.

Chapter 6, Advanced Integrations and Development, reviews modular inputs, data models, the KV store, and modular D3 visualizations.

Chapter 7, Packaging Applications, lists the items needed to package an App or add-on, getting it ready for publishing.

Chapter 8, Publishing Applications, describes step by step how to upload an App to Splunk base, and includes some information on Splunk's great support community.

What you need for this book

To take full advantage of all the examples and code contained in this book, you should have the following:

An installed and running instance of Splunk

Basic knowledge of how Splunk works, including searching, basic panels and dashboards

An understanding of the various technologies that Splunk uses. These include:

Python

JavaScript

HTML

CSS

Who this book is for

This book will benefit both the casual Splunker and the experienced professional alike. Whether you are just starting Splunk application or add-on development, or have been developing for years, this book has tips and tricks to help with developing new integrations, Splunk applications, and add-ons. Even for a quick modular input, this book provides quick tutorials on common integration techniques and code examples.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text are shown as follows: We can include other contexts through the use of the include directive.

A block of code is set as follows:

[bluecoat]

REPORT-extract = auto_kv_for_bluecoat

TIME_FORMAT = %b %d %Y

EVAL-app = bluecoat

\

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[bluecoat] REPORT-extract = auto_kv_for_bluecoat

 

TIME_FORMAT = %b %d %Y

EVAL-app = bluecoat

Any command-line input or output is written as follows:

# cp default/inputs.conf local/inputs.conf

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: Clicking on the Next button moves you to the next screen.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[e-mail]>, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any