This document explains how to get started using OpenStack Swift with Fog.
Start by executing the following command:
irb
Once irb
has launched you need to require the Fog library.
If using Ruby 1.8.x execute:
require 'rubygems' require 'fog'
If using Ruby 1.9.x execute:
require 'fog'
Next, create a connection to Swift:
service = Fog::Storage.new({ :provider => 'OpenStack', # OpenStack Fog provider :openstack_username => USERNAME, # Your OpenStack Username :openstack_api_key => PASSWORD, # Your OpenStack Password :openstack_auth_url => 'http://YOUR_OPENSTACK_ENDPOINT:PORT/v2.0/tokens' })
Alternative regions are specified using the key
:openstack_region
. A list of regions available for Swift can
be found by executing the following:
The Storage service supports the following additional parameters:
Key | Description |
---|---|
:persistent | If set to true, the service will use a persistent connection. |
:openstack_service_name | |
:openstack_service_type | |
:openstack_tenant | |
:openstack_region | |
:openstack_temp_url_key |
Fog supports passing additional
connection parameters to its underlying HTTP library (Excon) using the
:connection_options
parameter.
Key | Description |
---|---|
:connect_timeout | Connection timeout (default: 60 seconds) |
:write_timeout | Write timeout for connection (default: 60 seconds) |
:proxy | Proxy for HTTP and HTTPS connections |
:ssl_ca_path | Path to SSL certificate authorities |
:ssl_ca_file | SSL certificate authority file |
:ssl_verify_peer | SSL verify peer (default: true) |
Fog provides both a
model and request abstraction. The
request abstraction provides the most efficient interface and the model
abstraction wraps the request abstraction to provide a convenient
ActiveModel
like interface.
The Fog::Storage object supports a number of methods that wrap individual HTTP requests to the Swift API.
To see a list of requests supported by the storage service:
service.requests
This returns:
[:copy_object, :delete_container, :delete_object, :delete_multiple_objects, :delete_static_large_object, :get_container, :get_containers, :get_object, :get_object_http_url, :get_object_https_url, :head_container, :head_containers, :head_object, :put_container, :put_object, :put_object_manifest, :put_dynamic_obj_manifest, :put_static_obj_manifest, :post_set_meta_temp_url_key]
To request a view account details:
response = service.head_containers
This returns in the following Excon::Response
:
#<Excon::Response:0x10283fc68 @headers={"X-Account-Bytes-Used"=>"2563554", "Date"=>"Thu, 21 Feb 2013 21:57:02 GMT", "X-Account-Meta-Temp-Url-Key"=>"super_secret_key", "X-Timestamp"=>"1354552916.82056", "Content-Length"=>"0", "Content-Type"=>"application/json; charset=utf-8", "X-Trans-Id"=>"txe934924374a744c8a6c40dd8f29ab94a", "Accept-Ranges"=>"bytes", "X-Account-Container-Count"=>"7", "X-Account-Object-Count"=>"5"}, @status=204, @body="">
To view the status of the response:
response.status
Note: Fog is aware of the valid HTTP response statuses for each request type. If an unexpected HTTP response status occurs, Fog will raise an exception.
To view response headers:
response.headers
This will return:
{"X-Account-Bytes-Used"=>"2563554", "Date"=>"Thu, 21 Feb 2013 21:57:02 GMT", "X-Account-Meta-Temp-Url-Key"=>"super_secret_key", "X-Timestamp"=>"1354552916.82056", "Content-Length"=>"0", "Content-Type"=>"application/json; charset=utf-8", "X-Trans-Id"=>"txe934924374a744c8a6c40dd8f29ab94a", "Accept-Ranges"=>"bytes", "X-Account-Container-Count"=>"7", "X-Account-Object-Count"=>"5"}
To learn more about Fog::Storage
request methods refer to rdoc. To
learn more about Excon refer to Excon GitHub repo.
Fog models behave in a manner similar to
ActiveModel
. Models will generally respond to
create
, save
, destroy
,
reload
and attributes
methods. Additionally, fog
will automatically create attribute accessors.
Here is a summary of common model methods:
Method | Description |
---|---|
create |
Accepts hash of attributes and creates object. Note: creation is a non-blocking call and you will be required to wait for a valid state before using resulting object. |
save | Saves object. Note: not all objects support updating object. |
destroy |
Destroys object. Note: this is a non-blocking call and object deletion might not be instantaneous. |
reload | Updates object with latest state from service. |
attributes | Returns a hash containing the list of model attributes and values. | identity |
Returns the identity of the object. Note: This might not always be equal to object.id. |
The remainder of this document details the model abstraction.
Note: Fog sometimes refers to Swift containers as directories.
To retrieve a list of directories:
service.directories
This returns a collection of
Fog::Storage::OpenStack::Directory
models:
To retrieve a specific directory:
service.directories.get "blue"
This returns a Fog::Storage::OpenStack::Directory
instance:
To create a directory:
service.directories.create :key => 'backups'
The create
method also supports the following key values:
Key | Description |
---|---|
:metadata | Hash containing directory metadata. |
To delete a directory:
directory.destroy
Note: Directory must be empty before it can be deleted.
To list files in a directory:
directory.files
Note: File contents is not downloaded until
body
attribute is called.
To upload a file into a directory:
file = directory.files.create :key => 'space.jpg', :body => File.open "space.jpg"
Note: For files larger than 5 GB please refer to the Upload Large Files section.
The create
method also supports the following key values:
Key | Description |
---|---|
:content_type | The content type of the object. Cloud Files will attempt to auto detect this value if omitted. |
:access_control_allow_origin | URLs can make Cross Origin Requests. Format is http://www.example.com. Separate URLs with a space. An asterisk (*) allows all. Please refer to CORS Container Headers for more information. |
:origin | The origin is the URI of the object's host. |
:etag | The MD5 checksum of your object's data. If specified, Cloud Files will validate the integrity of the uploaded object. |
:metadata | Hash containing file metadata. |
Swift requires files larger than 5 GB (the Swift default limit) to be uploaded into segments along with an accompanying manifest file. All of the segments must be uploaded to the same container.
SEGMENT_LIMIT = 5368709119.0 # 5GB -1 BUFFER_SIZE = 1024 * 1024 # 1MB File.open(file_name) do |f| segment = 0 until file.eof? segment += 1 offset = 0 # upload segment to cloud files segment_suffix = segment.to_s.rjust(10, '0') service.put_object("my_container", "large_file/#{segment_suffix}", nil) do if offset <= SEGMENT_LIMIT - BUFFER_SIZE buf = file.read(BUFFER_SIZE).to_s offset += buf.size buf else '' end end end end # write manifest file service.put_object_manifest("my_container", "large_file", 'X-Object-Manifest' => "my_container/large_file/") # write manifest file service.put_object_manifest("my_container", "large_file", 'X-Object-Manifest' => "my_container/large_file/") # write manifest file service.put_object_manifest("my_container", "large_file", 'X-Object-Manifest' => "my_container/large_file/")
Segmented files are downloaded like ordinary files. See Download Files section for more information.
The most efficient way to download files from a private or public directory is as follows:
File.open('downloaded-file.jpg', 'w') do | f | directory.files.get("my_big_file.jpg") do | data, remaining, content_length | f.syswrite data end end
This will download and save the file in 1 MB chunks. The chunk size can be
changed by passing the parameter :chunk_size
into the
:connection_options
hash in the service constructor.
Note: The body
attribute of file will be
empty if a file has been downloaded using this method.
If a file object has already been loaded into memory, you can save it as follows:
File.open('germany.jpg', 'w') {|f| f.write(file_object.body) }
Note: This method is more memory intensive as the entire object is loaded into memory before saving the file as in the example above.
You can access metadata as an attribute on
Fog::Storage::Rackspace::File
.
file.metadata[:environment]
File metadata is set when the file is saved:
file.save
Metadata is reloaded when directory or file is reloaded:
file.reload
Cloud Files supports copying files. To copy files into a container named “trip” with a name of “europe.jpg” do the following:
file.copy("trip", "europe.jpg")
To move or rename a file, perform a copy operation and then delete the old file:
file.copy("trip", "germany.jpg") file.destroy
To delete a file:
file.destroy
Your feedback is appreciated! If you have specific issues with the fog SDK, you should file an issue via Github.