How to use the S3 command-line tool (Mac OSX/Linux)

The S3 command-line tool is the most reliable way of interacting with Amazon Web Services' Simple Storage Service (AWS S3). This article will guide you through setting it up and using it for common tasks.

Installing the tool

Open a terminal. On Mac OSX, the Terminal application can be found in Applications/Utilities.

Run the following commands:

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
./awscli-bundle/install -b ~/bin/aws

To verify that the tool is properly installed, run the following command
```
aws help
```

Configuring the tool

Follow the instructions in the AWS documentation under Quick Configuration. This will require you to input the access key ID and secret access key provided to you by DevOps.

Using the tool

List an S3 bucket directory:

Command

aws s3 ls s3://bucket-name/path/to/file

Response

						PRE dir1/
						PRE dir2/
2015-01-01 10:10:10    1234 File1.zip
2015-01-02 11:11:11 7654321 File2.pdf

This directory contains 4 things: 2 folders (named dir1 and dir2) and 2 files (File1.zip and File2.pdf). File1.zip was created on January 1, 2015 at 10:10:10 and is 1234 bytes large (roughly kilobytes). File2.pdf was created on January 2, 2015 at 11:11:11 and is 7654321 bytes large (about 7 megabytes).

Tip: Add the --human-readable flag to the end of your command to get more human-readable sizes

Command

aws s3 ls s3://bucket-name/path/to/file --human-readable

Response

						PRE dir1/
						PRE dir2/
2015-01-01 10:10:10 1.2 KiB File1.zip
2015-01-02 11:11:11 7.3 MiB File2.pdf

Download a file from S3 bucket:

aws s3 cp s3://bucket-name/path/to/file ~/Downloads

The file will be saved to your Downloads folder.

Upload a file to S3 bucket:

aws s3 cp ~/Downloads/file1.zip s3://bucket-name/path/to/destination --acl public-read

Tip

Instead of typing the path to the file on your computer, simply drag the file into the terminal.

Note

If you don't include –acl public-read, no one will be able to see your file. To see more options for setting permissions of file, use this link.

Remove a file from S3 bucket:

aws s3 rm s3://bucket-name/path/to/file

Sync your local and S3 bucket directories:

aws s3 sync . s3://mybucket

If you want to upload/download multiple files; just go to the directory where files are located and use above command giving the url of bucket folder where you want to upload. Can be done for the other way around (downloading), just swap the source and destination in the above command.

For more documentation, run aws s3 help or use aws docs.

Open edX Operations