Abstract

This document describes the Apache release signing and checksum policy, answers questions related to this policy and gives a basic primer on the wider subject. Links are provided to authoritative sources of deeper information.

Release managers are the target audience.

Contents

Help Wanted!

Help to finish this document by contributing documentation patches! If the information you seek isn't in this document, then please submit a patch once the infrastructure folks have answered your question.

Further Reading

Description Of Policy

Note this is not normative

Every artifact distributed by the Apache Software Foundation should and every new one must be accompanied by one file containing an OpenPGP compatible ASCII armored detached signature and another file containing an MD5 checksum. The names of these files must be formed by adding to the name of the artifact the following suffixes:

  • the signature by suffixing .asc
  • the checksum by suffixing .md5

An SHA checksum should also be created and must be suffixed .sha.

Release managers must not store private keys used to sign Apache releases on ASF hardware.

Notes For Projects

The KEYS File

The KEYS file is a plain text file containing the public key signatures of the release managers (and optionally other committers) for the project. For example, Apache Ant KEYS. It is traditional to include the following header (which explains its usage):

This file contains the PGP keys of various developers.

Users: pgp < KEYS
       gpg --import KEYS
Developers: 
        pgp -kxa <your name> and append it to this file.
        (pgpk -ll <your name> && pgpk -xa <your name>) >> this file.
        (gpg --list-sigs <your name>
             && gpg --armor --export <your name>) >> this file.

Note: this system will be replaced by a better process in the near future. In preparation, please ensure that public keys are connected as strongly as possible to the Apache web of trust and are available from the major public key servers.

The Basics

Applied cryptography is a subject that has considerable depth. Luckily, it's possible to get started signing releases without being an expert. Just remember that (from time to time) you will encounter situations that will require research and learning. Hopefully the FAQ will be a reasonable first port of call.

You will need an application to manage keys and create signatures. GNU Privacy Guard is recommended and this section assumes that's what you're using. Read the manual. Note that GnuPG can handle MD5 and SHA checksums as well as PGP signatures. It is your one-stop shop, cross-platform tool for release signing and verification.

It can be hard for newbies to be confident that they have executed operations correctly. Consider doing some practice first.

Why We Sign Releases

A signature allows anyone to verify that a file is identical to the one created by the Apache release manager. Using a signature:

  • users can make sure that what they received has not been modified in any way, either accidentally via a faulty transmission channel, or intentionally (with or without malicious intent)
  • the Apache infrastructure team can verify the identity of a file

OpenPGP signatures confer the usual advantages of digital signatures: authentication, integrity and non-repudiation. MD5 and SHA checksums only provide the integrity part as they are not encypted.

Security Basics
Key Basics

To sign releases, you need to generate a new master key-pair for code signing.

Generate a revocation certificate and store it somewhere safe.

Note the fingerprint of the new key. Consider noting the fingerprint and key ID on a public web page.

Your new public key should be uploaded to a major pgp keyserver.

Signing Basics

Signatures should be ASCII armored and detached.

Your public key should be exported and the result appended to the appropriate KEYS file(s).

That's all you need to know to sign a release.

Questions About Checksums And Signatures

How Do I Sign A Release?

Create a OpenPGP compatible ASCII armored detached signature for the released artifact. Upload the signature with the released artifact.

See here for a basic overview.

What Is an OpenPGP Compatible ASCII Armored Detached Signature?

It is an OpenPGP compatible ASCII armored detached signature.

To create one using GNU Privacy Guard for file foo.tar.gz type:

   $ gpg --armor --output foo.tar.gz.asc --detach-sig foo.tar.gz 
What Is OpenPGP?

OpenPGP is an RFC describing a system for interoperable public key cryptography. Various implementations exist. GNU Privacy Guard (GnuPG) is an open source OpenPGP compatible implementation. It comes with good documentation describing not just GnuPG but also giving a good general introduction to public key cryptography.

What Is Public Key Cryptography?

Public key cryptography is asymmetric. One key can be used to encrypt a message which only the other key can decrypt. Knowledge of the first key can be made public without compromising the security of the second key. One key is therefore called the public key and one the private key.

When using public key cryptography, the public key can be freely distributed but the private key must be secret. It is vital that private key files are protected. Private keys are typically stored in files protected by symmetric encryption. Strong passwords must be chosen to protect them.

What Is An Detached Signature?

A digital signature is created from an original document using a private key. Possession of the corresponding public key allows verification that a given file is identical to the original document. An attached signature is attached to the document whereas a detached signature is contained in a separate file.

What Is ASCII Armoring?

An encoding format that converts a binary file into a string of ASCII characters. This format is more human readable and more portable.

What Is An MD5 Checksum?

MD5 is a well known message digest algorithm.

Many tools are available to calculate these sums. For example, OpenSSL can be used:

$ openssl md5 < file

Platform specific applications are also common. For example md5sum on linux:

$ md5sum file

With GnuPG, your swiss army knife of release signing, you do

  $ gpg --print-md MD5 [fileName] > [fileName].md5
  
What is a SHA checksum?

Like MD5, SHA is another message digest algorithm. Using GnuPG, you can create a SHA signature as follows:

  $ gpg --print-md SHA1 [fileName] > [fileName].sha
  
What Is A Message Digest Algorithm?

A message digest algorithm takes a document and produces a much smaller hash of that document. A good algorithm will produce different digests for very similar documents. A good algorithm makes it infeasible to create a message matching a given hash.

A trusted digest for a document can be used to verify the contents of an untrusted file.

What Is A Web Of Trust?

It is difficult to personally verify the identity of all useful public keys. However, having verified the identity of only a small number of public keys it is possible to deduce the identity of public keys trusted by the owners of these keys. This process can be repeated. This extended graph of trusted identities is termed a web of trust.

Webs of trust can be used to solve the problem of verifying the identity of public keys.

Note: in order to take full advantage of a web of trust, it is important to actively build your web of trust into the major public webs of trust. Conferences are an ideal opportunity but you must come prepared.

For more information read Henk Penning's Apache home page and the GNU Privacy Guard User Guide.

By an existing member of that web of trust signing your public key to verify your identity. See Henk Penning's Key Signing HOWTO

In short, expect that:

  • this will involve a face-to-face meeting
  • that some form of identification will be demanded
  • you will be asked to verify their identity and sign their public key in exchange

The most effective way to achieve this is to attend a key signing party.

A small amount of preparation (before attending technical conferences or meetings) will allow keys to be exchanged (if the other person is suitably prepared) or your key signed if the opportunity presents itself. All that is required is suitable identification and the public key fingerprint (which can can be conveniently printed onto a small card).

What Is A Key Signing Party?

A key signing party is a meeting organised to allow the exchange of public keys and so extension of the web of trust.

See the Keysigning Party HOWTO and Henk Penning's Key Signing HOWTO

How Can I Link My Key Into The Apache Web of Trust?

By meeting other Apache committers face-to-face and exchanging public keys. There are several ways to achieve this:

  • Key signing parties are organised at each ApacheCon.
  • If you are not able to attend (or the conference is a long way off) then consider organising a face-to-face meeting of local committers.
  • Subscribe to the party list and when you visit a new city, see if committers want to meet up.

See Henk Penning's Key Signing HOWTO

What Does Verifying A Signature Mean?

Public key cryptography can be used to test whether a particular file is identical (in content) to an original by verifying a signature. The signature file is a digest of the original file signed by a public key which attests to the digest's authenticity.

For example, when using GNU Privacy Guard you verify the signature foo-1.0.tar.gz.asc for release foo-1.0.tar.gz using the following command:

$ gpg --verify foo-1.0.tar.gz.asc foo-1.0.tar.gz

Trust is required in the identity of the public key that made the signature and that the signature is for the original in question (and not some other file). When verifying a release from an untrusted source (for example, over P2P file sharing or from a mirror) it is therefore important to download the signature from a trusted source. Signatures for all Apache releases are available directly from www.apache.org and should be downloaded from there.

How Can I Check The Integrity Of A Release?

MD5 and SHA checksums provide a simple, means of verifying the integrity of a download. You can simply create a checksum (in the same way as the release manager) after download, and compare the result to the checksum downloaded from the main Apache site. Obviously, this process does not provide for authentication and non-repudiation as anybody can create the same checksum.

The integrity of a release can also be checked by verifying the signature. More knowledge is required to correctly interpret the result but it does provide authentication and non-repudiation. If you are connected to the Apache web of trust then this also offers superior security.

What Does 'Public Key Not Found' Mean (When Verifying A Signature)?

Before a signature can be verified, the public key is required.

For example, when using GNU Privacy Guard if you have never imported the appropriate public key a message similar to the following will be displayed:

$ gpg --verify foo-1.0.tar.gz.asc foo-1.0.tar.gz
gpg: Signature made Mon Sep 26 22:26:18 2005 BST using RSA key ID 00000000
gpg: Can't check signature: public key not found

Unknown keys can often be downloaded from public key servers. However, these should only be trusted through a web of trust.

Apache projects normally keep the developers' public keys in a file called KEYS. You may be able to find that file on the project's website, or in their code repository. Use

  $ gpg --import KEYS
  

to import the public keys.

What is a Trusted Key?

OpenPGP uses a web of trust. The owner of a public key who trusts the identity of a second key may mark this key as trusted by signing it. This has several major effects:

  • In future, no untrusted key warning will be issued when a valid signature for this key is verified.
  • Keys trusted by the owner of the key may also become trusted. In other words, a key whose identity has been confirmed by the owner of a key whose identity you trust may be automatically trusted. This behavior is typically configurable.
  • The next time you export your key, those who trust your key may start to trust the identity of the trusted key.

The transitive nature of the web of trust places a responsibility on the owner to verify the identity of the owner of those keys marked as trusted.

For more information read Henk Penning's Apache home page and the GNU Privacy Guard User Guide.

What Is The Difference Between A Valid Signature from an Untrusted Key And An Invalid Signature from an Untrusted Key?

Trustfulness and validity are different concepts. You may elect to trust the identity of a key to various degrees (or not at all). For a particular key, a particular signature for a particular file may be valid (in other words, created by the private key from an identical file) or invalid (either corrupt or created from a different file).

You should not trust a file with an invalid signature. You can trust a file with a valid signature as much as you trust the identity of key that was used to verify the signature.

For example, when using GNU Privacy Guard a message similar to the following indicates that the signature is invalid:

$ gpg --verify foo-1.0.tar.gz.asc foo-1.0.tar.gz
gpg: Signature made Mon Sep 26 22:26:18 2005 BST using RSA key ID 00000000
gpg: BAD signature from "someone@example.org"

whereas a message similar to the following indicates that the signature is valid but for an untrusted key:

$ gpg --verify foo-1.0.tar.gz.asc foo-1.0.tar.gz
gpg: Signature made Mon Sep 26 22:05:28 2005 BST using RSA key ID 00000000
gpg: Good signature from "someone@example.org"
gpg:                 aka "someone@anotherdomain.org"
gpg: checking the trustdb
gpg: checking at depth 0 signed=1 ot(-/q/n/m/f/u)=0/0/0/0/0/1
gpg: checking at depth 1 signed=0 ot(-/q/n/m/f/u)=1/0/0/0/0/0
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00

The fingerprint may be used to ascertain the appropriate level of trust to assign to the key.

What Is A Public Key Fingerprint?

Public keys are long and even when ASCII armored are not very easy for humans to understand or compare. A fingerprint is a shorter digest of the key formatted in a way that makes it easier for humans to read and compare.

Why Infeasible And Not Impossible?

Responsible cryptography talks about infeasible cracks (rather than impossible ones) since this is more accurate. All current practical methods can be subjected to brute force attacks and so can be cracked. So, a good question is whether attacks are feasible given the current state of the art.

Where Should I Create The Signatures?

Creating signatures requires the private key. Limited copies should be kept of the private key and all must be kept confidential. Though the file used to store the private key is typically protected by encryption, it is vulnerable to dictionary attacks on the passphrase. This file must therefore be kept secret. So, signatures should be created on the machine used to store the private key.

It is important that the private key is stored on secure hardware with limited read permissions and is protected by a good passphrase. Consider using removable media or an isolated installation.

A master private key used to sign Apache artifacts (or to secure communications with the ASF) is particularly valuable. If you want or need to be able to create signatures for other purposes (for example, signing email messages) in other (less secure) locations it is recommended that you create multiple sub keys and use sub keys for signing in less secure locations.

The private key must not be store on any ASF machine. So, signatures must not be created on ASF machines.

What Is 'Insecure Memory' And Should I Be Worried?

When you use GNU Privacy Guard you may see a warning similar to:

gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information

If you are using GnuPG on Apache hardware, please read this. Sensitive operations using a private key must not be executed on ASF hardware.

If you encounter this issue elsewhere then it indicates that GnuPG cannot lock memory pages which means they may be swapped out to disc. It would then be feasible for an attacker who had gained access to the machine to read the private key from the swap file. For more details, read the FAQ.

What is a Passphrase?

The term passphrase is often used in cryptography for what might be better known as a password in other contexts. For example, an OpenPGP private key is typically stored to disc in an file encrypted by a symmetric cypher keyed by a passphrase. This passphrase is one of the weakest elements in the system: should anyone else gain access to the file then a dictionary attack will be feasible on a weak passphrase. So, choosing a strong passphrase is very important.

Passphrases are (unlike passwords) typically unlimited in length. Long passphrases are recommended. This allows sequences of (at least seven) unrelated words to be used as well as more conventional mixtures of symbols and alphanumerics.

Note that even a good passphrase will offer only limited protection. Given the encrypted file and enough time, a determined cracker will be able to break any passphrase. A good passphrase will buy important time in the event of a compromise but is no substitute for keeping the private key safe and secure in the first place.

What Is A Revocation Certificate?

OpenPGP defines a special type of signed message called a revocation certificate. This message indicates that the signer believes that the key is no longer trustworthy. Typically, the revocation certificate will be signed by the key to be revoked (though the key may specify that other keys should be trusted for revocation). The type of revocation and the comment included may be used to judge how much trust to place in a good signature by a revoked key.

A revocation certificate should be generated for each public key used. These should be stored safely, securely and separately from the public key.

Each revocation certificates has a type specifying a general (machine readable) reason for the revocation:

  • No reason specified
  • Key has been compromised
  • Key is superseded
  • Key is no longer used

It is recommended that certificates are created to cover the first two cases. Note that if a key is lost or can no longer be accessed (due to media failure, say) it is best to assume that the key has been potentially compromised. It is recommended that revocation certificates are printed and stored safely to guard against media failure.

An ASCII armored revocation certificate for key bob can be generated and saved to revoke.asc using GNU Privacy Guard as follows:

$ gpg --output revoke.asc --armor --gen-revoke bob

The certificate produced should be securely stored.

If you are preparing a revocation certificate for future use, it is recommended that you test it. See safe practice.

How Do I Revoke A Key?

To revoke a key (given a certificate) using GNU Privacy Guard import the revocation certificate:

$ gpg --import revoke.asc 
gpg: key 4A03679A: "Some User <someuser@example.org>" revocation certificate imported
gpg: Total number processed: 1
gpg:    new key revocations: 1
Where Should A Revocation Certificate Be Stored?

The revocation certificate should be stored securely and separately from the key it revokes. Burning onto CDROM or printing out onto hard copy are good solutions.

How Do I Distribute A Revocation Certificate?

In the event of a compromise, a revocation certificate needs to be distributed to those using the key. This process needs to be a mirror of the process by which the original key was distributed.

  • The Apache infrastructure team should be informed by a post containing the revocation certificate
  • The KEYS files containing the original key should be updated with the revocation certificate
  • The revocation certificate should be uploaded to the major keyserver networks
  • An announcement should be posted to the appropriate lists with the revocation certificate attached
What Is The Difference Between Deleting And Revoking A Key?

When a key is deleted from a keyring, it is simply removed. It can be added again later.

When a key is revoked, the key is marked in the key ring. Whenever a message signed by this key is verified in the future, the user will be warned that the key has been revoked.

For example, when verifying a revoked key, GNU Privacy Guard issues the following comment:

$ gpg --verify message.asc .message 
gpg: Signature made Sat Apr  8 09:28:31 2006 BST using DSA key ID 4A03679A
gpg: Good signature from "Some User <someuser@example.org>"
gpg: checking the trustdb
gpg: checking at depth 0 signed=0 ot(-/q/n/m/f/u)=0/0/0/0/0/1
gpg: WARNING: This key has been revoked by its owner!
gpg:          This could mean that the signature is forgery.
gpg: reason for revocation: Key has been compromised
gpg: revocation comment: 
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 82D1 169B E6F1 9D14 DA76  A5DD 968E 66E4 4A03 679A
Can I Mark A Key As Locally Trusted?

On occasion, a key may be trusted by the user (who understands the risks) but is not considered trustworthy enough to be exported to the web of trust.

OpenPGP allows keys to be signed as local only. These trust relationships will not be exported to the public web of trust but will be treated as trusted when the key ring is used locally.

For example, with GNU Privacy Guard use:

$ gpg --lsign-key someuser
How Can I Safely Practice Using OpenPGP?

Use separate environments each with a different practice keyring.

For example, using GNU Privacy Guard:

  • (First time only) Create a directory to contain the keyring
  • Open the shell to be configured to use this keyring
  • Change to the directory
  • (First time only) $ mkdir -m 700 .gnupg
  • Setup environment $ export GNUPGHOME=.gnupg
What Is The Difference Between A Public And A Private Key?

A public key is used for verifying signatures and encrypting messages, a private key for generating signatures and decrypting messages. Public keys can be freely distributed safely whereas private keys must be kept protected. More details here.

How Should My Code Signing Private Key Be Protected?

Anyone who possesses a copy of a private key used to sign releases can create doctored releases with valid signatures. If this person intends harm then the consequences could be serious indeed.

It is therefore very important that this private key is kept secret.

How Secure Does The Machine Used To Sign Releases Need To Be?

If the code signing machine is owned then it is only a matter of time before the key is compromised.

At a minimum, the machine should well maintained (kept up to date with security patches, use appropriate anti-virus and firewall software). The ideal is an isolated, well maintained installation used only for creating releases. This can be achieved with a little effort by creating an isolated installation on a separate hard disc (which is physically disconnected when not in use signing releases) or a live CD.

Which Applications Create OpenPGP Compatible Signatures?

There are many applications available (some commercial, some freeware, some software libre). Whichever one you choose, please subscribe to the appropriate security lists and keep the application fully patched.

Here are some used by ASF release managers:

How Safe Does The Private Key Need To Be?

It is vital that the private key is kept safe and secure. Though the file is encrypted using a passphrase, given enough time any determined cracker will be able to break that encryption. It is therefore essential that the private key file is kept safe and secure.

Basic precautions should include ensuring that the directories are readable only by the user.

However, it is recommended that for code signing keys additional measures are taken. The window of opportunity can be reduced by either using a isolated installation or by storing the private key on removable media (which should be removed to secure storage when not being used to sign releases).

What Does 'Isolated Installation' Mean?

An installation which is inaccessible when not being used to sign releases. For example, create an installation on a separate hard disc or use a live CD.

What Key Length Is Recommended?

There is no good, simple answer to this question.

The number of operations required to break a key by brute force increases with key size. However, the cost of using the key also rises. So, the planned usage of the key must be a consideration. Keys used for code signing will only be used rarely and in situations where performance is not the main concern. This is a reason to err in favor of long key lengths.

Over time, the practice cost of attacking a key (of a given length) by brute force falls as computing power increases. So, a key whose length seems adequate today may be seem too short in a few years time. This is a significant issue for long-lived keys such as those used to sign ASF releases. Again, this is a reason to err in the favor of longer key lengths.

How Do You Generate A Code Signing Key?

The exact mechanics are application dependent. Please think about the right key length and choose a good passphrase.

To generate a new key using GNU Privacy Guard, an interactive key generation session can be initiated by:


$ gpg --gen-key

The defaults are usually reasonable (though the default key length may be a little short). It is recommended that the User-ID for the code signing key should by your Apache email address and that the comment should include CODE SIGNING KEY.

What Is A Public Key Server?

A public key server manages public keys. Available functions may vary but typically include upload, search and download.

Public key servers exist to distribute public keys. They do not vouch for the actual identity of the owner of each key. This must be established either directly or through a web of trust. Do not trust a key just because it has been downloaded from a key server.

The major public key servers synchronize their records regularly so a key uploaded to one should be disseminated to the rest.

Some well known public key servers:

How Do You Upload A Key To A Public Key Server?

There are two common ways to upload a key to a public key server:

For example using GNU Privacy Guard, the key with ID B1313DE2 can be exported to the default public key server by:

$ gpg --send-key B13131DE2

Note that each changed key must be exported separately.

How Can I Ensure My Local Web Of Trust Is Up To Date?

The public web of trust grows constantly as people sign new keys and upload the new signatures onto the network of public key servers. Public keys should be periodically refreshed to ensure that your local web of trust is as full as possible. Many OpenPGP clients allow keys to be easily refreshed by querying a public key server.

For example, to refresh all keys using GNU Privacy Guard use:

$ gpg --refresh-keys
How Do You Export A Key?

A public key can be exported using OpenPGP by using --export. Typically, the export should be ASCII armored. For example, to export all public keys to the command line use:


gpg --export --armor

In most cases, it is better to export all keys - this ensures that signatures made on other keys will be exported. However, it is possible to export just one key by specifying it on the command line.

Secret keys can also be exported. However, exporting secret keys poses a security risk and there are better solutions for most common use cases. For example, copying the GNUPGHOME directory (typically ~/.gnupg) is a better way to transfer an OpenPGP keyring from one machine to another.

What Is A Key ID?

A key ID is similar to a fingerprint but is much smaller in length. There is no guarantee that key IDs are unique. Consequently, it is strongly recommended that the fingerprint is checked before signing a key. The key ID is typically used for locating keys and identifying keys already contained within the keyring. For these use cases, key ID should be unique enough in practice.

What Is A Sub Key?

Each OpenPGP keyring has a single master key. This key is signing only. It may also optionally have a number of sub keys (for encryption and signing).

If you wish to sign emails using a key related to that used to sign code, it is recommended that a signing sub key is used.

How Do I A Use Sub Key To Sign Emails?

To keep a code signing key safe and secure it is recommended that the key is not kept on a hard disc on a regular development machine. This means that the master key should not be used directly to sign emails. However, there are occasions when digitally signed emails are desirable.

The recommended approach is to create a sub key for email signing and export it to the regular machine. The master key can then be kept safely offline. For more details, read:

Note that some public key servers do not handle sub keys correctly. It may be necessary to use one on the SKS network.

How Can I Find Out More?

See this.

Is There A Quick Way To Sign Several Distributions?

The private https://svn.apache.org/repos/private/committers repository contains scripts that assist with batch signing several distributions.