List all files in Amazon S3 bucket

[3970 views]




Suppose you have an amazon s3 bucket that has around hundreds or thousands of filenames in it. So what is the easiest way to get a text file that contains lists of all the filenames in that amazon s3 bucket?

List all files in Amazon S3 bucket

  1. Using AWS Command Line Interface (CLI)

    2. AWS have their own Command Line Tools. The AWS Command Line Interface (CLI) is a unified tool to manage all the AWS services. With just AWS CLI, you can control multiple AWS services from the command line and automate them by writing your own scripts.

    The AWS CLI can be installed on Linux, Windows, and macOS using pip, a package manager for Python that provides the most easy way to install, upgrade, and remove Python packages and their dependencies.

    If you already have pip installed and a supported version of Python, you can install the AWS CLI with the following command:

    $ pip install awscli --upgrade --user

    The --upgrade option tells pip to upgrade any requirements that are already installed. The --user option tells pip to install the program to a subdirectory of your user directory to avoid modifying libraries used by your operating system.

    You can also install AWS CLI using easy_install command

    sudo easy_install awscli

    Once you have simply installed AWS CLI you simply run this command

    aws s3 ls

    Above command will show you all of the available buckets

    CreationTime Bucket ------------ ------ 2017-08-20 12:06:40 mybucket1 2018-01-14 10:35:54 mybucket2 2018-03-18 19:44:32 mybucket3

    After this you can query a specific bucket for files like this

    aws s3 ls s3://mybucket2

    Output:

    Bucket: mybucket2 Prefix: LastWriteTime Length Name ------------- ------ ---- PRE someName/ 2018-03-18 19:44:32 88 myfile.txt

  2. Using boto

    3. boto is a Python interface to Amazon Web Services. For installing boto, visit boto's github page boto

    After installing boto you can simply write this python script and save it as a .py file

    bucketlist.py

    from boto.s3.connection import S3Connection conn = S3Connection('access-key','secret-access-key') bucket = conn.get_bucket('bucket') for key in bucket.list(): print key.name.encode('utf-8')

    Run this file on terminal

    $ python list.py > results.txt
                 

Updated on 25 FEBRUARY, 2021 by Shaddy




AI Search Engine:



Comments










Search Anything:

Sponsored Deals ends in


E-Books & Study Notes:






Technical Quizzes Specially For You:



Online Free Tools:
Search Tags
You Might Also Like