googleapis/java-storage

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Java idiomatic client for Cloud Storage.

MavenStability

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.61.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage</artifactId>
  </dependency>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage-control</artifactId>
  </dependency>
</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-storage</artifactId>
  <version>2.53.0</version>
</dependency>
<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-storage-control</artifactId>
  <version>2.53.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.61.0')

implementation 'com.google.cloud:google-cloud-storage'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-storage:2.53.0'

If you are using SBT, add this to your dependencies:

libraryDependencies += "com.google.cloud" % "google-cloud-storage" % "2.53.0"

See the Authentication section in the base directory's README.

The client application making API calls must be granted authorization scopes required for the desired Cloud Storage APIs, and the authenticated principal must have the IAM role(s) required to access GCP resources using the Cloud Storage API calls.

You will need a Google Cloud Platform Console project with the Cloud Storage API enabled. You will need to enable billing to use Google Cloud Storage. Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud Command Line Interface and running the following commands in command line: gcloud auth login and gcloud config set project [YOUR PROJECT ID].

You'll need to obtain the google-cloud-storage library. See the Quickstart section to add google-cloud-storage as a dependency in your code.

Cloud Storage is a durable and highly available object storage service. Google Cloud Storage is almost infinitely scalable and guarantees consistency: when a write succeeds, the latest copy of the object will be returned to any GET, globally.

See the Cloud Storage client library docs to learn how to use this Cloud Storage Client Library.

The Storage Control API lets you perform metadata-specific, control plane, and long-running operations.

The Storage Control API creates one space to perform metadata-specific, control plane, and long-running operations apart from the Storage API. Separating these operations from the Storage API improves API standardization and lets you run faster releases.

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>libraries-bom</artifactId>
            <version>26.44.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-storage-control</artifactId>
    </dependency>
</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage-control</artifactId>
    <version>2.41.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.44.0')
implementation 'com.google.cloud:google-cloud-storage-control'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-storage-control:2.41.0'

To make authenticated requests to Google Cloud Storage, you must create a service object with credentials. You can then make API calls by calling methods on the Storage service object. The simplest way to authenticate is to use Application Default Credentials. These credentials are automatically inferred from your environment, so you only need the following code to create your service object:

import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

Storage storage = StorageOptions.getDefaultInstance().getService();

For other authentication options, see the Authentication page in Google Cloud Java.

Stored objects are called "blobs" in google-cloud and are organized into containers called "buckets". Blob, a subclass of BlobInfo, adds a layer of service-related functionality over BlobInfo. Similarly, Bucket adds a layer of service-related functionality over BucketInfo. In this code snippet, we will create a new bucket and upload a blob to that bucket.

Add the following imports at the top of your file:

import static java.nio.charset.StandardCharsets.UTF_8;

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;

Then add the following code to create a bucket and upload a simple blob.

Important: Bucket names have to be globally unique (among all users of Cloud Storage). If you choose a bucket name that already exists, you'll get a helpful error message telling you to choose another name. In the code below, replace "my_unique_bucket" with a unique bucket name. See more about naming rules here.

// Create a bucket
String bucketName = "my_unique_bucket"; // Change this to something unique
Bucket bucket = storage.create(BucketInfo.of(bucketName));

// Upload a blob to the newly created bucket
BlobId blobId = BlobId.of(bucketName, "my_blob_name");
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
Blob blob = storage.create(blobInfo, "a simple blob".getBytes(UTF_8));

A complete example for creating a blob can be found at UploadObject.java.

At this point, you will be able to see your newly created bucket and blob on the Google Developers Console.

Now that we have content uploaded to the server, we can see how to read data from the server. Add the following line to your program to get back the blob we uploaded.

BlobId blobId = BlobId.of(bucketName, "my_blob_name");
byte[] content = storage.readAllBytes(blobId);
String contentString = new String(content, UTF_8);

A complete example for accessing blobs can be found at DownloadObject.java.

Another thing we may want to do is update a blob. The following snippet shows how to update a Storage blob if it exists.

BlobId blobId = BlobId.of(bucketName, "my_blob_name");
Blob blob = storage.get(blobId);
if (blob != null) {
  byte[] prevContent = blob.getContent();
  System.out.println(new String(prevContent, UTF_8));
  WritableByteChannel channel = blob.writer();
  channel.write(ByteBuffer.wrap("Updated content".getBytes(UTF_8)));
  channel.close();
}

Suppose that you've added more buckets and blobs, and now you want to see the names of your buckets and the contents of each one. Add the following code to list all your buckets and all the blobs inside each bucket.

// List all your buckets
System.out.println("My buckets:");
for (Bucket bucket : storage.list().iterateAll()) {
  System.out.println(bucket);

  // List all blobs in the bucket
  System.out.println("Blobs in the bucket:");
  for (Blob blob : bucket.list().iterateAll()) {
    System.out.println(blob);
  }
}

See ListObjects.java for a complete example.

  • Bookshelf - An App Engine application that manages a virtual bookshelf.
    • This app uses google-cloud to interface with Cloud Datastore and Cloud Storage. It also uses Cloud SQL, another Google Cloud Platform service.
  • Flexible Environment/Storage example - An app that uploads files to a public Cloud Storage bucket on the App Engine Flexible Environment runtime.

Samples are in the samples/ directory.

SampleSource CodeTry it
Configure Retriessource codeOpen in Cloud Shell
Generate Signed Post Policy V4source codeOpen in Cloud Shell
Get Service Accountsource codeOpen in Cloud Shell
Quickstart Grpc Dp Samplesource codeOpen in Cloud Shell
Quickstart Grpc Samplesource codeOpen in Cloud Shell
Quickstart Open Telemetry Samplesource codeOpen in Cloud Shell
Quickstart Samplesource codeOpen in Cloud Shell
Quickstart Storage Control Samplesource codeOpen in Cloud Shell
Add Bucket Default Ownersource codeOpen in Cloud Shell
Add Bucket Iam Conditional Bindingsource codeOpen in Cloud Shell
Add Bucket Iam Membersource codeOpen in Cloud Shell
Add Bucket Labelsource codeOpen in Cloud Shell
Add Bucket Ownersource codeOpen in Cloud Shell
Change Default Storage Classsource codeOpen in Cloud Shell
Configure Bucket Corssource codeOpen in Cloud Shell
Create Bucketsource codeOpen in Cloud Shell
Create Bucket Dual Regionsource codeOpen in Cloud Shell
Create Bucket Pub Sub Notificationsource codeOpen in Cloud Shell
Create Bucket With Object Retentionsource codeOpen in Cloud Shell
Create Bucket With Storage Class And Locationsource codeOpen in Cloud Shell
Create Bucket With Turbo Replicationsource codeOpen in Cloud Shell
Delete Bucketsource codeOpen in Cloud Shell
Delete Bucket Pub Sub Notificationsource codeOpen in Cloud Shell
Disable Bucket Versioningsource codeOpen in Cloud Shell
Disable Default Event Based Holdsource codeOpen in Cloud Shell
Disable Lifecycle Managementsource codeOpen in Cloud Shell
Disable Requester Payssource codeOpen in Cloud Shell
Disable Soft Deletesource codeOpen in Cloud Shell
Disable Uniform Bucket Level Accesssource codeOpen in Cloud Shell
Enable Bucket Versioningsource codeOpen in Cloud Shell
Enable Default Event Based Holdsource codeOpen in Cloud Shell
Enable Lifecycle Managementsource codeOpen in Cloud Shell
Enable Requester Payssource codeOpen in Cloud Shell
Enable Uniform Bucket Level Accesssource codeOpen in Cloud Shell
Get Bucket Autoclasssource codeOpen in Cloud Shell
Get Bucket Metadatasource codeOpen in Cloud Shell
Get Bucket Rposource codeOpen in Cloud Shell
Get Default Event Based Holdsource codeOpen in Cloud Shell
Get Public Access Preventionsource codeOpen in Cloud Shell
Get Requester Pays Statussource codeOpen in Cloud Shell
Get Retention Policysource codeOpen in Cloud Shell
Get Soft Delete Policysource codeOpen in Cloud Shell
Get Uniform Bucket Level Accesssource codeOpen in Cloud Shell
List Bucket Iam Memberssource codeOpen in Cloud Shell
List Bucketssource codeOpen in Cloud Shell
List Pub Sub Notificationssource codeOpen in Cloud Shell
Lock Retention Policysource codeOpen in Cloud Shell
Make Bucket Publicsource codeOpen in Cloud Shell
Print Bucket Aclsource codeOpen in Cloud Shell
Print Bucket Acl Filter By Usersource codeOpen in Cloud Shell
Print Pub Sub Notificationsource codeOpen in Cloud Shell
Remove Bucket Corssource codeOpen in Cloud Shell
Remove Bucket Default Kms Keysource codeOpen in Cloud Shell
Remove Bucket Default Ownersource codeOpen in Cloud Shell
Remove Bucket Iam Conditional Bindingsource codeOpen in Cloud Shell
Remove Bucket Iam Membersource codeOpen in Cloud Shell
Remove Bucket Labelsource codeOpen in Cloud Shell
Remove Bucket Ownersource codeOpen in Cloud Shell
Remove Retention Policysource codeOpen in Cloud Shell
Set Async Turbo Rposource codeOpen in Cloud Shell
Set Bucket Autoclasssource codeOpen in Cloud Shell
Set Bucket Default Kms Keysource codeOpen in Cloud Shell
Set Bucket Website Infosource codeOpen in Cloud Shell
Set Client Endpointsource codeOpen in Cloud Shell
Set Default Rposource codeOpen in Cloud Shell
Set Public Access Prevention Enforcedsource codeOpen in Cloud Shell
Set Public Access Prevention Inheritedsource codeOpen in Cloud Shell
Set Retention Policysource codeOpen in Cloud Shell
Set Soft Delete Policysource codeOpen in Cloud Shell
Create Foldersource codeOpen in Cloud Shell
Create Hierarchical Namespace Bucketsource codeOpen in Cloud Shell
Delete Foldersource codeOpen in Cloud Shell
Get Foldersource codeOpen in Cloud Shell
List Folderssource codeOpen in Cloud Shell
Rename Foldersource codeOpen in Cloud Shell
Activate Hmac Keysource codeOpen in Cloud Shell
Create Hmac Keysource codeOpen in Cloud Shell
Deactivate Hmac Keysource codeOpen in Cloud Shell
Delete Hmac Keysource codeOpen in Cloud Shell
Get Hmac Keysource codeOpen in Cloud Shell
List Hmac Keyssource codeOpen in Cloud Shell
Create Managed Foldersource codeOpen in Cloud Shell
Delete Managed Foldersource codeOpen in Cloud Shell
Get Managed Foldersource codeOpen in Cloud Shell
List Managed Folderssource codeOpen in Cloud Shell
Add Blob Ownersource codeOpen in Cloud Shell
Batch Set Object Metadatasource codeOpen in Cloud Shell
Change Object Csek To Kmssource codeOpen in Cloud Shell
Change Object Storage Classsource codeOpen in Cloud Shell
Compose Objectsource codeOpen in Cloud Shell
Copy Objectsource codeOpen in Cloud Shell
Copy Old Version Of Objectsource codeOpen in Cloud Shell
Delete Objectsource codeOpen in Cloud Shell
Delete Old Version Of Objectsource codeOpen in Cloud Shell
Download Byte Rangesource codeOpen in Cloud Shell
Download Encrypted Objectsource codeOpen in Cloud Shell
Download Objectsource codeOpen in Cloud Shell
Download Object Into Memorysource codeOpen in Cloud Shell
Download Public Objectsource codeOpen in Cloud Shell
Download Requester Pays Objectsource codeOpen in Cloud Shell
Generate Encryption Keysource codeOpen in Cloud Shell
Generate V4 Get Object Signed Urlsource codeOpen in Cloud Shell
Generate V4 Put Object Signed Urlsource codeOpen in Cloud Shell
Get Object Metadatasource codeOpen in Cloud Shell
List Objectssource codeOpen in Cloud Shell
List Objects With Old Versionssource codeOpen in Cloud Shell
List Objects With Prefixsource codeOpen in Cloud Shell
List Soft Deleted Objectssource codeOpen in Cloud Shell
List Soft Deleted Versions Of Objectsource codeOpen in Cloud Shell
Make Object Publicsource codeOpen in Cloud Shell
Move Objectsource codeOpen in Cloud Shell
Print Blob Aclsource codeOpen in Cloud Shell
Print Blob Acl For Usersource codeOpen in Cloud Shell
Release Event Based Holdsource codeOpen in Cloud Shell
Release Temporary Holdsource codeOpen in Cloud Shell
Remove Blob Ownersource codeOpen in Cloud Shell
Restore Soft Deleted Objectsource codeOpen in Cloud Shell
Rotate Object Encryption Keysource codeOpen in Cloud Shell
Set Event Based Holdsource codeOpen in Cloud Shell
Set Object Metadatasource codeOpen in Cloud Shell
Set Object Retention Policysource codeOpen in Cloud Shell
Set Temporary Holdsource codeOpen in Cloud Shell
Stream Object Downloadsource codeOpen in Cloud Shell
Stream Object Uploadsource codeOpen in Cloud Shell
Upload Encrypted Objectsource codeOpen in Cloud Shell
Upload Kms Encrypted Objectsource codeOpen in Cloud Shell
Upload Objectsource codeOpen in Cloud Shell
Upload Object From Memorysource codeOpen in Cloud Shell
Allow Divide And Conquer Downloadsource codeOpen in Cloud Shell
Allow Parallel Composite Uploadsource codeOpen in Cloud Shell
Download Bucketsource codeOpen in Cloud Shell
Download Manysource codeOpen in Cloud Shell
Upload Directorysource codeOpen in Cloud Shell
Upload Manysource codeOpen in Cloud Shell

To get help, follow the instructions in the shared Troubleshooting document.

Cloud Storage uses HTTP/JSON for the transport layer.

Java 8 or above is required for using this client.

Google's Java client libraries, Google Cloud Client Libraries and Google Cloud API Libraries, follow the Oracle Java SE support roadmap (see the Oracle Java SE Product Releases section).

In general, new feature development occurs with support for the lowest Java LTS version covered by Oracle's Premier Support (which typically lasts 5 years from initial General Availability). If the minimum required JVM for a given library is changed, it is accompanied by a semver major release.

Java 11 and (in September 2021) Java 17 are the best choices for new development.

Google tests its client libraries with all current LTS versions covered by Oracle's Extended Support (which typically lasts 8 years from initial General Availability).

Google's client libraries support legacy versions of Java runtimes with long term stable libraries that don't receive feature updates on a best efforts basis as it may not be possible to backport all es.

Google provides updates on a best efforts basis to apps that continue to use Java 7, though apps might need to upgrade to current versions of the library that supports their JVM.

The latest versions and the supported Java versions are identified on the individual repository .com/GoogleAPIs/java-SERVICENAME and on google-cloud-java.

This library follows Semantic Versioning, but does update Storage interface to introduce new methods which can break your implementations if you implement this interface for testing purposes.

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

Apache 2.0 - See LICENSE for more information.

Java VersionStatus
Java 8Kokoro CI
Java 8 OSXKokoro CI
Java 8 WindowsKokoro CI
Java 11Kokoro CI

Java is a registered trademark of Oracle and/or its affiliates.

About

No description, website, or topics provided.

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Packages

No packages published

Contributors 79

Languages