ModSecurity 2.5

By Magnus Mischel

This book teaches ModSecurity from the beginning to anyone with basic Linux skills. It starts by focusing on introducing ModSecurity, and explaining the concept of ModSecurity rules and how to write them. Later, it looks at the performance of ModSecurity and what sort of impact ModSecurity has on the speed of your web application. This book is written for system administrators or anyone running an Apache web server who wants to learn how to secure that server. It assumes that you are familiar with using the Linux shell and command-line tools, but does its best to explain everything so that those who are not Linux experts can make full use of ModSecurity.

• Language: English
• Publisher: Packt Publishing
• Release date: Nov 23, 2009
• ISBN: 9781847194756

Book preview

Table of Contents

ModSecurity 2.5

Credits

About the Author

About the Reviewers

Preface

What ModSecurity is

Why you need ModSecurity

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Errata

Piracy

Questions

1. Installation and Configuration

Versions

Downloading

Checking the integrity of the downloaded source archive

Unpacking the source code

Required additional libraries and files

Compilation

Integrating ModSecurity with Apache

Configuration file

Completing the configuration

Testing your installation

Creating a simple ModSecurity rule

Disguising the web server signature

Summary

2. Writing Rules

SecRule syntax

Variables and collections

The transaction collection

Storing data between requests

Examining several variables

Quotes: Sometimes you need them and sometimes you don't

Creating chained rules

Rule IDs

An introduction to regular expressions

Examples of regular expressions

More about regular expressions

Using @rx to block a remote host

Simple string matching

Matching numbers

More about collections

Counting items in collections

Filtering collection fields using a regular expression

Built-in fields

Transformation functions

Other operators

Set-based pattern matching with @pm and @pmFromFile

@pmFromFile

Performance of the phrase matching operators

Validating character ranges

Phases and rule ordering

Actions—what to do when a rule matches

Allowing requests

Blocking requests

Taking no action but continuing rule processing

Dropping requests

Redirecting and proxying requests

SecAction

Using the ctl action to control the rule engine

How to use the ctl action

Macro expansion

SecRule in practice

Blocking uncommon request methods

Restricting access to certain times of day

Detecting credit card leaks

Detecting credit card numbers

The Luhn algorithm and false positives

Tracking the geographical location of your visitors

GEO collection fields

Blocking users from specific countries

Load balancing requests between servers on different continents

Pausing requests for a specified amount of time

Executing shell scripts

Sending alert emails

Sending more detailed alert emails

Counting file downloads

Blocking brute-force password guessing

Injecting data into responses

Inspecting uploaded files

Summary

3. Performance

A typical HTTP request

A real-world performance test

The core ruleset

Installing the core ruleset

Making sure it works

Performance testing basics

Using httperf

Getting a baseline: Testing without ModSecurity

Response time

Memory usage

CPU usage

ModSecurity without any loaded rules

ModSecurity with the core ruleset loaded

Response time

Memory usage

Finding the bottleneck

Wrapping up core ruleset performance

Optimizing performance

Memory consumption

Bypassing inspection of static content

Using @pm and @pmFromFile

Logging

Writing regular expressions for best performance

Use non-capturing parentheses wherever possible

Use one regular expression whenever possible

Summary

4. Audit Logging

Enabling the audit log engine

Single versus multiple file logging

Determining what to log

The configuration so far

Log format

Concurrent logging

Selectively disabling logging

Audit log sanitization actions

The ModSecurity Console

Installing the ModSecurity Console

Accessing the Console

Compiling mlogc

Configuring mlogc

Forwarding logs to the ModSecurity Console

Summary

5. Virtual Patching

Why use virtual patching?

Speed

Stability

Flexibility

Cost-effectiveness

Creating a virtual patch

From vulnerability discovery to virtual patch: An example

Creating the patch

Changing the web application for additional security

Testing your patches

Real-life examples

Geeklog

Patching Geeklog

Cross-site scripting

Real-life example: The Twitter worm

Summary

6. Blocking Common Attacks

HTTP fingerprinting

How HTTP fingerprinting works

Server banner

Response header

HTTP protocol responses

Issuing an HTTP DELETE request

Bad HTTP version numbers

Bad protocol name

The ETag HTTP header

Using ModSecurity to defeat HTTP fingerprinting

Blocking proxied requests

Cross-site scripting

Preventing XSS attacks

PDF XSS protection

HttpOnly cookies to prevent XSS attacks

Session identifiers

Cross-site request forgeries

Protecting against cross-site request forgeries

Shell command execution attempts

Null byte attacks

ModSecurity and null bytes

Source code revelation

Directory traversal attacks

Blog spam

SQL injection

Standard injection attempts

Retrieving data from multiple tables with UNION

Multiple queries in one call

Reading arbitrary files

Writing data to files

Preventing SQL injection attacks

What to block

Website defacement

Brute force attacks

Directory indexing

Detecting the real IP address of an attacker

Summary

7. Chroot Jails

What is a chroot jail?

A sample attack

Traditional chrooting

How ModSecurity helps jailing Apache

Using ModSecurity to create a chroot jail

Verifying that the jail works

Chroot caveats

Summary

8. REMO

More about Remo

Installation

Remo rules

Creating and editing rules

Installing the rules

Analyzing log files

Configuration tweaks

Summary

9. Protecting a Web Application

Considerations before beginning

The web application

Groundwork

Step 1: Identifying user actions

Step 2: Getting detailed information on each action

Step 3: Writing rules

Step 4: Testing the new ruleset

Actions

Blocking what's allowed—denying everything else

Cookies

Headers

Securing the Start New Topic action

The ruleset so far

The finished ruleset

Alternative approaches

Keeping everything up to date

Summary

A. Directives and Variables

Directives

SecAction

SecArgumentSeparator

SecAuditEngine

SecAuditLog

SecAuditLog2

SecAuditLogParts

SecAuditLogRelevantStatus

SecAuditLogStorageDir

SecAuditLogType

SecCacheTransformations (deprecated/experimental)

SecChrootDir

SecComponentSignature

SecContentInjection

SecCookieFormat

SecDataDir

SecDebugLog

SecDebugLogLevel

SecDefaultAction

SecGeoLookupDb

SecGuardianLog

SecMarker

SecPdfProtect

SecPdfProtectMethod

SecPdfProtectSecret

SecPdfProtectTimeout

SecPdfProtectTokenName

SeqRequestBodyAccess

SecRequestBodyLimit

SecRequestBodyNoFilesLimit

SecRequestBodyInMemoryLimit

SecResponseBodyLimit

SecResponseBodyLimitAction

SecResponseBodyMimeType

SecResponseBodyMimeTypesClear

SecResponseBodyAccess

SecRule

SecRuleInheritance

SecRuleEngine

SecRuleRemoveById

SecRuleRemoveByMsg

SecRuleUpdateActionById

SecServerSignature

SecTmpDir

SecUploadDir

SecUploadFileMode

SecUploadKeepFiles

SecWebAppId

Variables

ARGS

ARGS_COMBINED_SIZE

ARGS_NAMES

ARGS_GET

ARGS_GET_NAMES

ARGS_POST

ARGS_POST_NAMES

AUTH_TYPE

ENV

FILES

FILES_COMBINED_SIZE

FILES_NAMES

FILES_SIZES

FILES_TMPNAMES

GEO

HIGHEST_SEVERITY

MATCHED_VAR

MATCHED_VAR_NAME

MODSEC_BUILD

MULTIPART_CRLF_LF_LINES

MULTIPART_STRICT_ERROR

MULTIPART_UNMATCHED_BOUNDARY

PATH_INFO

QUERY_STRING

REMOTE_ADDR

REMOTE_HOST

REMOTE_PORT

REMOTE_USER

REQBODY_PROCESSOR

REQBODY_PROCESSOR_ERROR

REQBODY_PROCESSOR_ERROR_MSG

REQUEST_BASENAME

REQUEST_BODY

REQUEST_COOKIES

REQUEST_COOKIES_NAMES

REQUEST_FILENAME

REQUEST_HEADERS

REQUEST_HEADERS_NAMES

REQUEST_LINE

REQUEST_METHOD

REQUEST_PROTOCOL

REQUEST_URI

REQUEST_URI_RAW

RESPONSE_BODY

RESPONSE_CONTENT_LENGTH

RESPONSE_CONTENT_TYPE

RESPONSE_HEADERS

RESPONSE_HEADERS_NAMES

RESPONSE_PROTOCOL

RESPONSE_STATUS

RULE

SCRIPT_BASENAME

SCRIPT_FILENAME

SCRIPT_GID

SCRIPT_GROUPNAME

SCRIPT_MODE

SCRIPT_UID

SCRIPT_USERNAME

SERVER_ADDR

SERVER_NAME

SERVER_PORT

SESSION

SESSIONID

TIME

TIME_DAY

TIME_EPOCH

TIME_HOUR

TIME_MIN

TIME_MON

TIME_SEC

TIME_WDAY

TIME_YEAR

TX

USERID

WEBAPPID

WEBSERVER_ERROR_LOG

XML

B. Regular Expressions

What is a regular expression?

Regular expression flavors

Example of a regular expression

Identifying an email address

The Dot character

Quantifiers—star, plus, and question mark

Question Mark

Star

Plus sign

Grouping

Ranges

Alternation

Backreferences

Captures and ModSecurity

Non-capturing parentheses

Character classes

Negated matching

Shorthand notation

Anchors

Start and end of string

Word Boundary

Lazy quantifiers

Debugging regular expressions

Additional resources

Our email address regex

Summary

Index

ModSecurity 2.5

Securing your Apache installation and web applications

Magnus Mischel

---

ModSecurity 2.5

Securing your Apache installation and web applications

Copyright © 2009 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: November 2009

Production Reference: 1171109

Published by Packt Publishing Ltd.

32 Lincoln Road

Olton

Birmingham, B27 6PA, UK.

ISBN 978-1-847194-74-9

www.packtpub.com

Cover Image by Karl Moore (<[e-mail]> )

Credits

Author

Magnus Mischel

Reviewers

Kai 'Oswald' Seidler

Daniel Cuthbert

Acquisition Editor

James Lumsden

Development Editor

Dhiraj Chandiramani

Technical Editor

Conrad Sardinha

Copy Editor

Sanchari Mukherjee

Indexer

Rekha Nair

Editorial Team Leader

Gagandeep Singh

Project Team Leader

Lata Basantani

Project Coordinator

Rajashree Hamine

Proofreader

Lynda Sliwoski

Graphics

Nilesh Mohite

Production Coordinator

Aparna Bhagat

Cover Work

Aparna Bhagat

About the Author

Magnus Mischel is the founder and director of Mischel Internet Security (http://www.misec.net), whose product TrojanHunter helps protect computers against malware. His long-time passion for computer security is what lead to him starting the company after realizing the threat that trojans and other malware pose to users. He currently lives in London, and when he isn't writing books or managing the company, he enjoys playing a game of chess at the Metropolitan Chess Club. He holds an MSc in Computer Science and Engineering from Linköping University, Sweden.

About the Reviewers

Kai 'Oswald' Seidler was born in Hamburg in 1970. He graduated from Technical University of Berlin with a Diplom Informatiker degree (Master of Science equivalent) in Computer Science. In the 90's he created and managed Germany's biggest IRCnet server irc.fu-berlin.de, and co-managed one of the world's largest anonymous FTP server ftp.cs.tu-berlin.de. He professionally set up his first public web server in 1993. From 1993 until 1998 he was member of Projektgruppe Kulturraum Internet, a research project on net culture and network organization. In 2002, he co-founded Apache Friends and created the multi-platform Apache web server bundle XAMPP. Around 2005 XAMPP became the most popular Apache stack worldwide. In 2006, his third book, Das XAMPP-Handbuch, was published by Addison Wesley.

Currently he's working as technology evangelist for web tier products at Sun Microsystems.

Daniel Cuthbert heads up Corsaire's Security Training and has over nine years of industry experience. During this time he has focused on Security Assessment for some of the world's largest consultancies and financial, telecommunication, and media institutions.

He holds a Masters Degree from the University of Westminster in IT Security and is both a founding member of the Open Web Application Security Project (OWASP) and previous UK Chapter Head. He has worked on helping companies adopt the Secure Development Lifecycle (SDLC) approach and has lectured extensively on the subject.

He has worked on a wide variety of books for the OWASP project.

Preface

Running a web service leaves you exposed in a lot of different ways. A security vulnerability in the web server software or any of the additional modules needed to run the site can result in a compromised server, lost data, and downtime. As a compromised server costs a lot to restore in terms of time and resources it pays to have the best security possible in place, and ModSecurity is an essential tool to protect your web service. This book aims to show you how to protect your server using ModSecurity as an important layer to prevent intrusions and detect intrusion attempts.

What ModSecurity is

ModSecurity is a web application firewall. Much like a traditional firewall, it filters incoming and outgoing data and is able to stop traffic that is considered malicious according to a set of predefined rules. It also has many advanced features such as HTTP transaction logging and content injection, which we will be covering later.

Rules are created and edited using a simple text format, which affords you great flexibility in writing your own rules. Once you master the syntax of ModSecurity rules you will be able to quickly write your own rules to block a new exploit or stop a vulnerability being taken advantage of. Make no mistake though, this text-based rule language is also very powerful and allows you to create very advanced filters as we will see in the later chapters.

Think of ModSecurity as a customs agent that sits at the border. Every request is examined to make sure no unauthorised payloads make it inside, to your web server. When an attack is discovered, the details can be written to a log file, or an email can be sent out to the administrator of the site to alert of the attempted intrusion.

Why you need ModSecurity

Imagine that you have your web server set up and chugging along nicely. It is serving requests without problems and not even straining under the load. Suddenly, it stops working—the web server port is closed and you can't even log on to it via SSH.

What you don't know is that there is a 0-day exploit for PHP, which you happen to be running on the server since most of the dynamic content is written in PHP. A malicious hacker has managed to use this new exploit to execute shell code on your server that added a new user account and allowed him to log on via SSH. Once inside, he proceeded to use a well-known privilege elevation exploit for the old version of the Linux kernel you are running to gain root privileges. As root he then had total control of the server and decided to gather whatever data he could off the server before panicking and deciding to cover his tracks by wiping the server's hard drive.

Does this sound far-fetched? It's not. Most compromised servers are attacked by using the web service to gain initial entry to it. You can't exploit a closed port, and port 80 is the one port that needs to be open to everyone for a web server to be able to function. Once inside, the attacker can then use other exploits to gain additional privileges, as in this example.

The difficulty in protecting a server is that new exploits appear all the time, and even if you patch them within a few days (which is something very few sites do!), there is still a certain time window where you are vulnerable to being attacked.

ModSecurity allows you to protect your server by writing generic rules that cover a broad range of possible attacking scenarios. Thus, ModSecurity is an additional layer that can protect you in a way that no patching, no matter how swift or meticulously done, can.

What this book covers

Chapter 1: Installation and Configuration shows how to compile ModSecurity from source, install and integrate it with Apache, and make sure it works.

Chapter 2: Writing Rules teaches you everything you need to know about writing ModSecurity rules.

Chapter 3: Performance takes a look at the performance of ModSecurity and what impact, if any, it has on the speed of your server and web application.

Chapter 4: Logging and Auditing teaches you how to configure logging and how to use the ModSecurity console to view logs online.

Chapter 5: Virtual Patching covers the technique of creating a virtual patch to fix any vulnerability which does not have a vendor-supplied patch, or where the source code to the web application is not available or easily patched.

Chapter 6: Blocking Common Attacks explains how common attacks on the web today work, and how to block them using ModSecurity.

Chapter 7: Chroot Jails is about creating a chroot jail for Apache, and how this can easily be accomplished using ModSecurity (usually it is a quite tedious task).

Chapter 8: REMO teaches you how to install and use the Rule Editor for ModSecurity (REMO), which is a graphical tool to create ModSecurity rules.

Chapter 9: Securing a Web Application takes a real-life web application and secures it using a positive security model, which means that only requests that correspond to a pre-defined pattern are allowed through; anything else is denied.

Appendix A: Directives and Variables contains a list of the directives available for use in your ModSecurity configuration file and also the variables available for use in rule writing.

Appendix B: Regular Expressions teaches you the basics of regular expressions so that you can make use of them when writing ModSecurity rules in a better way.

What you need for this book

This book is mainly targeted at Linux systems and as such most of the commands will be Linux commands. Many systems today run standard configurations such as LAMP (Linux, Apache, MySQL, PHP) and the book will put focus on those setups that are commonly used in real-world environments.

ModSecurity runs on many other platforms, such as FreeBSD, OpenBSD, HP-UX and Mac OS X. If you are familiar with the differences between Linux and your platform you should be able to use the advice in this book to get everything working on your particular platform.

As of the release date of this book the latest version of ModSecurity is 2.5. You can always find the latest release at www.modsecurity.org, which is the project's official web site.

Who this book is for

This book is aimed at the web server administrator who wishes to install and use ModSecurity on one or several web servers; either his own or those used by a company. The book does not assume the reader is an expert in Internet security and thus most vulnerabilities and exploits will be explained so that the reader is better able to understand the threat and the reason to guard against it.

There are many articles available online that cover ModSecurity; however most of them only examine one or two aspects of the module such as installation or how to write rules. This book aims to be a complete guide to the process of installing and deploying the module. You can also use this book as a reference guide when you need to create rules for a new or existing web server.

Once finished with the book, you will have a better idea of the exploits that are currently used by malicious hackers, and you will also know how to protect your servers against these and other exploits.

Some of the rules are aimed at specific application setups or languages such as PHP or SQL. However the book aims to explain the reason for creating the rules in as general terms as possible so that even readers who are not familiar with these languages will understand why certain rules will protect the server against attack.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text are shown as follows: Some collections have fixed fields, such as the GEO collection, which contains fields such as COUNTRY_NAME and CITY.

A block of code is set as follows:

SecRule REQUEST_URI passwd pass,setvar:tx.hackscore=+5

SecRule REQUEST_URI pass,setvar:tx.hackscore=+10

SecRule TX:HACKSCORE @gt 10 deny

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

--e8d98139-B-- GET /login.php?password=****** HTTP/1.1

 

Host: bytelayer.com

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

Cookie: JSESSIONID=4j4gl8be129l6

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: If necessary, you can change the protocol used to standard HTTP in the console settings under Administration | Web Server Configuration once you have logged in.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an email to <[e-mail]>, and mention the book title via the subject of your message.

If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or email .

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book on, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Tip

Downloading the example code for the book

Visit http://www.packtpub.com/files/code/4749_Code.zip to directly download the example code.

The downloadable files contain instructions on how to use them.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration, and help us to improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the let us know link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata added to any list of existing errata. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or web site name immediately so that we can pursue a remedy.

Please contact us at <[e-mail]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <[e-mail]> if you are having a problem with any aspect of the book, and we will do our best to address it.

Chapter 1. Installation and Configuration

This chapter deals with the installation and basic configuration of ModSecurity. In this chapter you will learn about the following, among other things:

Which additional libraries are required to compile ModSecurity

How to compile ModSecurity from source

How to integrate ModSecurity with Apache

Getting the basic configuration for ModSecurity in place

Testing that ModSecurity is working correctly

If you're new to ModSecurity I would recommend that you set it up on a test server while you get acquainted with the ins and outs of using it. You wouldn't want to deploy it on a production server only to find out a few days later that you've been blocking everyone in Europe from accessing your server because of a misconfiguration. (For more on blocking users