The intention of this article is to describe how to store files in cloud storage using the Golang stowutils package.
How does it work?
Implementations are referred to by a kind of string (like s3, google) and an object which provides specific configuration values associated with the implementation (like the S3 region, the credentials, the project ID in case of Google Cloud, etc.)
It uses stow package.
Stow provides implementations for cloud storage services. All providers almost have a similar approach when it comes to storing content files (or items) and are stored in buckets (or containers).
Object storage services are the concept around which the idea of stow is structured. It is made up of three main objects:
- Location – The Container objects are stored here.
- Container – Group of Item objects.
- Item – an individual file.
location1 (e.g. s3) container1 item1.1 item1.2 container2 item2.1 item2.2 location2 (e.g. google) container1 item1.1 item1.2 container2 item2.1 item2.2
It should be noted that:
- Location is like s3 or Google, and each location may have many containers (or storage buckets).
- A container contains many items ( or files).
- Containers do not contain other containers.
- Items must belong to a container.
Pre-requisite:
Import Stowutils and implementations:
First, you import stowutils and any implementations you wish to support:
import ( utils "github.com/agiratech/stowutils" )
Features of the Stowutils package:
- Get Location
- Acquire list of containers
- Get Container
- Acquire list of Items
- Get Item
- Upload Item
- Read Item
- Download Item
- Delete Item
Get Location
In order to get a location, you need to know the kind of string and a utils.Config object that contains the required configuration information (such as AccessKey, secretKey, JSON Credentials, projectID, etc). Configuration implementation is specific, and so, you should consult each implementation to see what fields are required for particular kind.
Example: For google cloud storage
g_location, err := utils.GetLocation("google",&utils.Config{
GoogleConfigJSONFilePath:"json-credentials.json",
GoogleProjectID:"sample-project"})
if err != nil {
return err
}
defer g_location.Close()
// use g_location
Example: For s3 cloud storage
s_location, err := utils.GetLocation("s3", &utils.Config{
S3AccessKeyID:"1452468515",
S3SecretKey:"a1b2c3d4",
S3Region:"us-west-2"})
if err != nil {
return err
}
defer s_location.Close()
// use s_location
Get list of containers
You can get Container list using the utils.ContainerList function:
func ContainerList(location stow.Location, prefix string, pageSize int) ([]stow.Container, error)
Here, prefix is a prefix name of containers.
Example:
g_containers, err := utils.ContainerList(g_location, "", 100)
if err != nil {
return err
}
for _,c := range g_containers {
// use container c
}
Get Container
You can get a required container using the utils.GetContainer function:
func GetContainer(location stow.Location, container_name string, pageSize int)
(stow.Container, error) {
Here, container_name is a required container name (and hence the container_name should be available)
Example:
g_container, err := utils.GetContainer(g_location, "sample-container", 100)
if err != nil {
return err
}
// use container
Get a list of Items
You can get item list using utils.ItemList function:
func ItemList(container stow.Container, prefix string, pageSize int) ([]stow.Item, error) {
Here, prefix is the prefix name of items.
Example:
g_items, err := utils.ItemList(g_container, "", 100)
if err != nil {
return err
}
for _,i := range g_items {
// use item i
}
Get Item
You can get a particular item using utils.GetItem function:
func GetItem(container stow.Container, item_name string, pageSize int) (stow.Item, error) {
The item_name mentioned here is a required item name (and hence the item_name should be available)
Example:
g_item, err := utils.GetItem(g_container, "img024.jpg", 100)
if err != nil {
return err
}
// use item
Upload Item
You can upload an item(file) using utils.UploadItem function:
func UploadItem(container stow.Container, file_path string, file_name string, metadata
map[string]interface{}) (stow.Item, error) {
Into whichever container you want to upload a file, pass that container as parameter. Here, file_path is an uploading file path and file_name is the name of the file, which should be unique. Otherwise, it will override an existing file.
Example:
item, err := utils.UploadItem(g_container, "/home/images/image001.jpg", "image001.jpg", nil)
if err != nil {
return err
}
// check the item
Read Item
You can read the item information using utils.ReadItem function:
func ReadItem(g_item stow.Item) (io.ReadCloser, error) {
Here, g_item is a received item using utils.GetItem
Example:
file, err = utils.ReadItem(item)
if err != nil {
return err
}
// use file
Download Item
Also, you can get the URL of the file using utils.DownloadItem function:
func DownloadItem(item stow.Item) *url.URL {
url = utils.DownloadItem(item) // use URL
Delete Item
You can delete an item from a container using utils.DeleteItem function as mentioned below:
func DeleteItem(container stow.Container, item_name string) error {
Pass that container and file_name into the function as a params (the file_name should be available), which contains the file that you want to delete.
Example:
err = utils.DeleteItem(g_container, item.Name())
if err != nil {
return err
}
Conclusion
The above mentioned are all the features of the Stowutils package, and this article covers the basics of how it can be done, and what are the possibilities with it in Golang. Thus, the Stowutils package can be used to achieve the storing of files in the Cloud Storage. Try it out yourself for a better understanding.
We offer Golang development services for building world-class enterprise apps. We have expertise in building most complex software solutions using Google’s Go language. Chat with us now and hire golang developers within 72 hours.
