GnuPG is a complete and free implementation of the OpenPGP standard as defined by RFC4880 (also known as PGP). GnuPG allows to encrypt and sign your data and communication, features a versatile key management system as well as access modules for all kinds of public key directories.
Install the software for the OS where you will be exchanging secure email. Webmail (Google apps, gmail, yahoo, etc) users should use the "Web Browser" plugin Mailvelope.
Each GPG user has a unique cryptographic key pair, which includes a public version and a private version.
To send and receive secure messages using GPG, you must generate your key pair. Software specific documentation:
Linux users: Generate and Share a Public Key
Windows users: http://gpg4win.de/handbuecher/novices_6.html
Web Browser: https://www.mailvelope.com/en/help#keys
Most GPG implementations allow you to easily publish your public key on a key server. This will allow people who know your email address to find your key and send you encrypted email. The most popular key server at EdX is pgp.mit.edu, but there are many others.