argo_navis: venv/lib/python2.7/site-packages/boto/s3/key.py annotate

author	bcclaywell
date	Mon, 12 Oct 2015 17:43:33 -0400
parents
children

rev	line source
0 d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	1 # Copyright (c) 2006-2012 Mitch Garnaat http://garnaat.org/
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	2 # Copyright (c) 2011, Nexenta Systems Inc.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	3 # Copyright (c) 2012 Amazon.com, Inc. or its affiliates. All Rights Reserved
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	4 #
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	5 # Permission is hereby granted, free of charge, to any person obtaining a
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	6 # copy of this software and associated documentation files (the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	7 # "Software"), to deal in the Software without restriction, including
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	8 # without limitation the rights to use, copy, modify, merge, publish, dis-
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	9 # tribute, sublicense, and/or sell copies of the Software, and to permit
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	10 # persons to whom the Software is furnished to do so, subject to the fol-
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	11 # lowing conditions:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	12 #
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	13 # The above copyright notice and this permission notice shall be included
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	14 # in all copies or substantial portions of the Software.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	15 #
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	16 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	17 # OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	18 # ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	19 # SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	20 # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	21 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	22 # IN THE SOFTWARE.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	23 import email.utils
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	24 import errno
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	25 import hashlib
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	26 import mimetypes
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	27 import os
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	28 import re
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	29 import base64
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	30 import binascii
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	31 import math
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	32 from hashlib import md5
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	33 import boto.utils
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	34 from boto.compat import BytesIO, six, urllib, encodebytes
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	35
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	36 from boto.exception import BotoClientError
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	37 from boto.exception import StorageDataError
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	38 from boto.exception import PleaseRetryException
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	39 from boto.provider import Provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	40 from boto.s3.keyfile import KeyFile
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	41 from boto.s3.user import User
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	42 from boto import UserAgent
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	43 from boto.utils import compute_md5, compute_hash
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	44 from boto.utils import find_matching_headers
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	45 from boto.utils import merge_headers_by_name
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	46
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	47
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	48 class Key(object):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	49 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	50 Represents a key (object) in an S3 bucket.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	51
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	52 :ivar bucket: The parent :class:`boto.s3.bucket.Bucket`.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	53 :ivar name: The name of this Key object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	54 :ivar metadata: A dictionary containing user metadata that you
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	55 wish to store with the object or that has been retrieved from
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	56 an existing object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	57 :ivar cache_control: The value of the `Cache-Control` HTTP header.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	58 :ivar content_type: The value of the `Content-Type` HTTP header.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	59 :ivar content_encoding: The value of the `Content-Encoding` HTTP header.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	60 :ivar content_disposition: The value of the `Content-Disposition` HTTP
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	61 header.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	62 :ivar content_language: The value of the `Content-Language` HTTP header.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	63 :ivar etag: The `etag` associated with this object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	64 :ivar last_modified: The string timestamp representing the last
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	65 time this object was modified in S3.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	66 :ivar owner: The ID of the owner of this object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	67 :ivar storage_class: The storage class of the object. Currently, one of:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	68 STANDARD \| REDUCED_REDUNDANCY \| GLACIER
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	69 :ivar md5: The MD5 hash of the contents of the object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	70 :ivar size: The size, in bytes, of the object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	71 :ivar version_id: The version ID of this object, if it is a versioned
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	72 object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	73 :ivar encrypted: Whether the object is encrypted while at rest on
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	74 the server.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	75 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	76
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	77 DefaultContentType = 'application/octet-stream'
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	78
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	79 RestoreBody = """<?xml version="1.0" encoding="UTF-8"?>
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	80 <RestoreRequest xmlns="http://s3.amazonaws.com/doc/2006-03-01">
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	81 <Days>%s</Days>
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	82 </RestoreRequest>"""
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	83
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	84
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	85 BufferSize = boto.config.getint('Boto', 'key_buffer_size', 8192)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	86
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	87 # The object metadata fields a user can set, other than custom metadata
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	88 # fields (i.e., those beginning with a provider-specific prefix like
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	89 # x-amz-meta).
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	90 base_user_settable_fields = set(["cache-control", "content-disposition",
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	91 "content-encoding", "content-language",
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	92 "content-md5", "content-type",
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	93 "x-robots-tag", "expires"])
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	94 _underscore_base_user_settable_fields = set()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	95 for f in base_user_settable_fields:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	96 _underscore_base_user_settable_fields.add(f.replace('-', '_'))
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	97 # Metadata fields, whether user-settable or not, other than custom
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	98 # metadata fields (i.e., those beginning with a provider specific prefix
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	99 # like x-amz-meta).
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	100 base_fields = (base_user_settable_fields \|
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	101 set(["last-modified", "content-length", "date", "etag"]))
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	102
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	103
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	104
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	105 def __init__(self, bucket=None, name=None):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	106 self.bucket = bucket
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	107 self.name = name
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	108 self.metadata = {}
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	109 self.cache_control = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	110 self.content_type = self.DefaultContentType
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	111 self.content_encoding = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	112 self.content_disposition = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	113 self.content_language = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	114 self.filename = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	115 self.etag = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	116 self.is_latest = False
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	117 self.last_modified = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	118 self.owner = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	119 self._storage_class = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	120 self.path = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	121 self.resp = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	122 self.mode = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	123 self.size = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	124 self.version_id = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	125 self.source_version_id = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	126 self.delete_marker = False
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	127 self.encrypted = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	128 # If the object is being restored, this attribute will be set to True.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	129 # If the object is restored, it will be set to False. Otherwise this
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	130 # value will be None. If the restore is completed (ongoing_restore =
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	131 # False), the expiry_date will be populated with the expiry date of the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	132 # restored object.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	133 self.ongoing_restore = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	134 self.expiry_date = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	135 self.local_hashes = {}
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	136
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	137 def __repr__(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	138 if self.bucket:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	139 name = u'<Key: %s,%s>' % (self.bucket.name, self.name)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	140 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	141 name = u'<Key: None,%s>' % self.name
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	142
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	143 # Encode to bytes for Python 2 to prevent display decoding issues
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	144 if not isinstance(name, str):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	145 name = name.encode('utf-8')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	146
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	147 return name
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	148
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	149 def __iter__(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	150 return self
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	151
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	152 @property
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	153 def provider(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	154 provider = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	155 if self.bucket and self.bucket.connection:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	156 provider = self.bucket.connection.provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	157 return provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	158
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	159 def _get_key(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	160 return self.name
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	161
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	162 def _set_key(self, value):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	163 self.name = value
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	164
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	165 key = property(_get_key, _set_key);
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	166
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	167 def _get_md5(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	168 if 'md5' in self.local_hashes and self.local_hashes['md5']:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	169 return binascii.b2a_hex(self.local_hashes['md5'])
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	170
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	171 def _set_md5(self, value):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	172 if value:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	173 self.local_hashes['md5'] = binascii.a2b_hex(value)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	174 elif 'md5' in self.local_hashes:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	175 self.local_hashes.pop('md5', None)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	176
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	177 md5 = property(_get_md5, _set_md5);
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	178
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	179 def _get_base64md5(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	180 if 'md5' in self.local_hashes and self.local_hashes['md5']:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	181 md5 = self.local_hashes['md5']
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	182 if not isinstance(md5, bytes):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	183 md5 = md5.encode('utf-8')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	184 return binascii.b2a_base64(md5).decode('utf-8').rstrip('\n')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	185
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	186 def _set_base64md5(self, value):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	187 if value:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	188 if not isinstance(value, six.string_types):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	189 value = value.decode('utf-8')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	190 self.local_hashes['md5'] = binascii.a2b_base64(value)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	191 elif 'md5' in self.local_hashes:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	192 del self.local_hashes['md5']
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	193
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	194 base64md5 = property(_get_base64md5, _set_base64md5);
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	195
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	196 def _get_storage_class(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	197 if self._storage_class is None and self.bucket:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	198 # Attempt to fetch storage class
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	199 list_items = list(self.bucket.list(self.name.encode('utf-8')))
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	200 if len(list_items) and getattr(list_items[0], '_storage_class',
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	201 None):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	202 self._storage_class = list_items[0]._storage_class
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	203 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	204 # Key is not yet saved? Just use default...
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	205 self._storage_class = 'STANDARD'
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	206
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	207 return self._storage_class
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	208
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	209 def _set_storage_class(self, value):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	210 self._storage_class = value
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	211
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	212 storage_class = property(_get_storage_class, _set_storage_class)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	213
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	214 def get_md5_from_hexdigest(self, md5_hexdigest):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	215 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	216 A utility function to create the 2-tuple (md5hexdigest, base64md5)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	217 from just having a precalculated md5_hexdigest.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	218 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	219 digest = binascii.unhexlify(md5_hexdigest)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	220 base64md5 = encodebytes(digest)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	221 if base64md5[-1] == '\n':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	222 base64md5 = base64md5[0:-1]
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	223 return (md5_hexdigest, base64md5)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	224
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	225 def handle_encryption_headers(self, resp):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	226 provider = self.bucket.connection.provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	227 if provider.server_side_encryption_header:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	228 self.encrypted = resp.getheader(
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	229 provider.server_side_encryption_header, None)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	230 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	231 self.encrypted = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	232
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	233 def handle_version_headers(self, resp, force=False):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	234 provider = self.bucket.connection.provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	235 # If the Key object already has a version_id attribute value, it
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	236 # means that it represents an explicit version and the user is
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	237 # doing a get_contents_*(version_id=<foo>) to retrieve another
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	238 # version of the Key. In that case, we don't really want to
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	239 # overwrite the version_id in this Key object. Comprende?
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	240 if self.version_id is None or force:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	241 self.version_id = resp.getheader(provider.version_id, None)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	242 self.source_version_id = resp.getheader(provider.copy_source_version_id,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	243 None)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	244 if resp.getheader(provider.delete_marker, 'false') == 'true':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	245 self.delete_marker = True
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	246 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	247 self.delete_marker = False
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	248
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	249 def handle_restore_headers(self, response):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	250 provider = self.bucket.connection.provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	251 header = response.getheader(provider.restore_header)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	252 if header is None:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	253 return
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	254 parts = header.split(',', 1)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	255 for part in parts:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	256 key, val = [i.strip() for i in part.split('=')]
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	257 val = val.replace('"', '')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	258 if key == 'ongoing-request':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	259 self.ongoing_restore = True if val.lower() == 'true' else False
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	260 elif key == 'expiry-date':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	261 self.expiry_date = val
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	262
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	263 def handle_addl_headers(self, headers):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	264 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	265 Used by Key subclasses to do additional, provider-specific
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	266 processing of response headers. No-op for this base class.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	267 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	268 pass
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	269
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	270 def open_read(self, headers=None, query_args='',
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	271 override_num_retries=None, response_headers=None):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	272 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	273 Open this key for reading
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	274
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	275 :type headers: dict
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	276 :param headers: Headers to pass in the web request
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	277
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	278 :type query_args: string
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	279 :param query_args: Arguments to pass in the query string
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	280 (ie, 'torrent')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	281
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	282 :type override_num_retries: int
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	283 :param override_num_retries: If not None will override configured
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	284 num_retries parameter for underlying GET.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	285
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	286 :type response_headers: dict
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	287 :param response_headers: A dictionary containing HTTP
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	288 headers/values that will override any headers associated
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	289 with the stored object in the response. See
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	290 http://goo.gl/EWOPb for details.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	291 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	292 if self.resp is None:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	293 self.mode = 'r'
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	294
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	295 provider = self.bucket.connection.provider
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	296 self.resp = self.bucket.connection.make_request(
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	297 'GET', self.bucket.name, self.name, headers,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	298 query_args=query_args,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	299 override_num_retries=override_num_retries)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	300 if self.resp.status < 199 or self.resp.status > 299:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	301 body = self.resp.read()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	302 raise provider.storage_response_error(self.resp.status,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	303 self.resp.reason, body)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	304 response_headers = self.resp.msg
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	305 self.metadata = boto.utils.get_aws_metadata(response_headers,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	306 provider)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	307 for name, value in response_headers.items():
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	308 # To get correct size for Range GETs, use Content-Range
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	309 # header if one was returned. If not, use Content-Length
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	310 # header.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	311 if (name.lower() == 'content-length' and
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	312 'Content-Range' not in response_headers):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	313 self.size = int(value)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	314 elif name.lower() == 'content-range':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	315 end_range = re.sub('./(.)', '\\1', value)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	316 self.size = int(end_range)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	317 elif name.lower() in Key.base_fields:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	318 self.__dict__[name.lower().replace('-', '_')] = value
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	319 self.handle_version_headers(self.resp)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	320 self.handle_encryption_headers(self.resp)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	321 self.handle_restore_headers(self.resp)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	322 self.handle_addl_headers(self.resp.getheaders())
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	323
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	324 def open_write(self, headers=None, override_num_retries=None):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	325 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	326 Open this key for writing.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	327 Not yet implemented
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	328
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	329 :type headers: dict
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	330 :param headers: Headers to pass in the write request
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	331
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	332 :type override_num_retries: int
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	333 :param override_num_retries: If not None will override configured
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	334 num_retries parameter for underlying PUT.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	335 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	336 raise BotoClientError('Not Implemented')
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	337
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	338 def open(self, mode='r', headers=None, query_args=None,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	339 override_num_retries=None):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	340 if mode == 'r':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	341 self.mode = 'r'
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	342 self.open_read(headers=headers, query_args=query_args,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	343 override_num_retries=override_num_retries)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	344 elif mode == 'w':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	345 self.mode = 'w'
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	346 self.open_write(headers=headers,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	347 override_num_retries=override_num_retries)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	348 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	349 raise BotoClientError('Invalid mode: %s' % mode)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	350
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	351 closed = False
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	352
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	353 def close(self, fast=False):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	354 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	355 Close this key.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	356
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	357 :type fast: bool
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	358 :param fast: True if you want the connection to be closed without first
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	359 reading the content. This should only be used in cases where subsequent
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	360 calls don't need to return the content from the open HTTP connection.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	361 Note: As explained at
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	362 http://docs.python.org/2/library/httplib.html#httplib.HTTPConnection.getresponse,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	363 callers must read the whole response before sending a new request to the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	364 server. Calling Key.close(fast=True) and making a subsequent request to
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	365 the server will work because boto will get an httplib exception and
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	366 close/reopen the connection.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	367
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	368 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	369 if self.resp and not fast:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	370 self.resp.read()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	371 self.resp = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	372 self.mode = None
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	373 self.closed = True
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	374
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	375 def next(self):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	376 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	377 By providing a next method, the key object supports use as an iterator.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	378 For example, you can now say:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	379
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	380 for bytes in key:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	381 write bytes to a file or whatever
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	382
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	383 All of the HTTP connection stuff is handled for you.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	384 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	385 self.open_read()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	386 data = self.resp.read(self.BufferSize)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	387 if not data:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	388 self.close()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	389 raise StopIteration
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	390 return data
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	391
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	392 # Python 3 iterator support
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	393 __next__ = next
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	394
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	395 def read(self, size=0):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	396 self.open_read()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	397 if size == 0:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	398 data = self.resp.read()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	399 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	400 data = self.resp.read(size)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	401 if not data:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	402 self.close()
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	403 return data
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	404
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	405 def change_storage_class(self, new_storage_class, dst_bucket=None,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	406 validate_dst_bucket=True):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	407 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	408 Change the storage class of an existing key.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	409 Depending on whether a different destination bucket is supplied
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	410 or not, this will either move the item within the bucket, preserving
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	411 all metadata and ACL info bucket changing the storage class or it
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	412 will copy the item to the provided destination bucket, also
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	413 preserving metadata and ACL info.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	414
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	415 :type new_storage_class: string
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	416 :param new_storage_class: The new storage class for the Key.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	417 Possible values are:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	418 * STANDARD
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	419 * REDUCED_REDUNDANCY
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	420
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	421 :type dst_bucket: string
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	422 :param dst_bucket: The name of a destination bucket. If not
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	423 provided the current bucket of the key will be used.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	424
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	425 :type validate_dst_bucket: bool
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	426 :param validate_dst_bucket: If True, will validate the dst_bucket
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	427 by using an extra list request.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	428 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	429 bucket_name = dst_bucket or self.bucket.name
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	430 if new_storage_class == 'STANDARD':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	431 return self.copy(bucket_name, self.name,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	432 reduced_redundancy=False, preserve_acl=True,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	433 validate_dst_bucket=validate_dst_bucket)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	434 elif new_storage_class == 'REDUCED_REDUNDANCY':
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	435 return self.copy(bucket_name, self.name,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	436 reduced_redundancy=True, preserve_acl=True,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	437 validate_dst_bucket=validate_dst_bucket)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	438 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	439 raise BotoClientError('Invalid storage class: %s' %
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	440 new_storage_class)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	441
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	442 def copy(self, dst_bucket, dst_key, metadata=None,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	443 reduced_redundancy=False, preserve_acl=False,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	444 encrypt_key=False, validate_dst_bucket=True):
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	445 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	446 Copy this Key to another bucket.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	447
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	448 :type dst_bucket: string
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	449 :param dst_bucket: The name of the destination bucket
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	450
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	451 :type dst_key: string
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	452 :param dst_key: The name of the destination key
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	453
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	454 :type metadata: dict
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	455 :param metadata: Metadata to be associated with new key. If
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	456 metadata is supplied, it will replace the metadata of the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	457 source key being copied. If no metadata is supplied, the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	458 source key's metadata will be copied to the new key.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	459
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	460 :type reduced_redundancy: bool
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	461 :param reduced_redundancy: If True, this will force the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	462 storage class of the new Key to be REDUCED_REDUNDANCY
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	463 regardless of the storage class of the key being copied.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	464 The Reduced Redundancy Storage (RRS) feature of S3,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	465 provides lower redundancy at lower storage cost.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	466
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	467 :type preserve_acl: bool
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	468 :param preserve_acl: If True, the ACL from the source key will
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	469 be copied to the destination key. If False, the
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	470 destination key will have the default ACL. Note that
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	471 preserving the ACL in the new key object will require two
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	472 additional API calls to S3, one to retrieve the current
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	473 ACL and one to set that ACL on the new object. If you
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	474 don't care about the ACL, a value of False will be
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	475 significantly more efficient.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	476
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	477 :type encrypt_key: bool
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	478 :param encrypt_key: If True, the new copy of the object will
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	479 be encrypted on the server-side by S3 and will be stored
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	480 in an encrypted form while at rest in S3.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	481
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	482 :type validate_dst_bucket: bool
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	483 :param validate_dst_bucket: If True, will validate the dst_bucket
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	484 by using an extra list request.
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	485
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	486 :rtype: :class:`boto.s3.key.Key` or subclass
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	487 :returns: An instance of the newly created key object
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	488 """
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	489 dst_bucket = self.bucket.connection.lookup(dst_bucket,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	490 validate_dst_bucket)
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	491 if reduced_redundancy:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	492 storage_class = 'REDUCED_REDUNDANCY'
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	493 else:
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	494 storage_class = self.storage_class
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	495 return dst_bucket.copy_key(dst_key, self.bucket.name,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	496 self.name, metadata,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	497 storage_class=storage_class,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	498 preserve_acl=preserve_acl,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	499 encrypt_key=encrypt_key,
d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b bcclaywell parents: diff changeset	500 src_version_id=self.version_id)

0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

4 #

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

5 # Permission is hereby granted, free of charge, to any person obtaining a

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

6 # copy of this software and associated documentation files (the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

7 # "Software"), to deal in the Software without restriction, including

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

8 # without limitation the rights to use, copy, modify, merge, publish, dis-

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

9 # tribute, sublicense, and/or sell copies of the Software, and to permit

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

10 # persons to whom the Software is furnished to do so, subject to the fol-

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

11 # lowing conditions:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

12 #

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

13 # The above copyright notice and this permission notice shall be included

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

14 # in all copies or substantial portions of the Software.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

15 #

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

16 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

17 # OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

18 # ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

19 # SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

20 # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

21 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

22 # IN THE SOFTWARE.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

23 import email.utils

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

24 import errno

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

25 import hashlib

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

26 import mimetypes

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

27 import os

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

28 import re

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

29 import base64

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

30 import binascii

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

31 import math

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

32 from hashlib import md5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

33 import boto.utils

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

34 from boto.compat import BytesIO, six, urllib, encodebytes

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

35

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

36 from boto.exception import BotoClientError

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

37 from boto.exception import StorageDataError

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

38 from boto.exception import PleaseRetryException

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

39 from boto.provider import Provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

40 from boto.s3.keyfile import KeyFile

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

41 from boto.s3.user import User

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

42 from boto import UserAgent

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

43 from boto.utils import compute_md5, compute_hash

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

44 from boto.utils import find_matching_headers

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

45 from boto.utils import merge_headers_by_name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

46

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

47

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

48 class Key(object):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

49 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

50 Represents a key (object) in an S3 bucket.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

51

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

52 :ivar bucket: The parent :class:`boto.s3.bucket.Bucket`.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

53 :ivar name: The name of this Key object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

54 :ivar metadata: A dictionary containing user metadata that you

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

55 wish to store with the object or that has been retrieved from

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

56 an existing object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

57 :ivar cache_control: The value of the `Cache-Control` HTTP header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

58 :ivar content_type: The value of the `Content-Type` HTTP header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

59 :ivar content_encoding: The value of the `Content-Encoding` HTTP header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

60 :ivar content_disposition: The value of the `Content-Disposition` HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

61 header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

62 :ivar content_language: The value of the `Content-Language` HTTP header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

63 :ivar etag: The `etag` associated with this object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

64 :ivar last_modified: The string timestamp representing the last

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

65 time this object was modified in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

66 :ivar owner: The ID of the owner of this object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

67 :ivar storage_class: The storage class of the object. Currently, one of:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

68 STANDARD | REDUCED_REDUNDANCY | GLACIER

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

69 :ivar md5: The MD5 hash of the contents of the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

70 :ivar size: The size, in bytes, of the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

71 :ivar version_id: The version ID of this object, if it is a versioned

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

72 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

73 :ivar encrypted: Whether the object is encrypted while at rest on

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

74 the server.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

75 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

76

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

77 DefaultContentType = 'application/octet-stream'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

78

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

79 RestoreBody = """<?xml version="1.0" encoding="UTF-8"?>

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

80 <RestoreRequest xmlns="http://s3.amazonaws.com/doc/2006-03-01">

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

81 <Days>%s</Days>

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

82 </RestoreRequest>"""

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

83

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

84

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

85 BufferSize = boto.config.getint('Boto', 'key_buffer_size', 8192)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

86

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

87 # The object metadata fields a user can set, other than custom metadata

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

88 # fields (i.e., those beginning with a provider-specific prefix like

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

89 # x-amz-meta).

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

90 base_user_settable_fields = set(["cache-control", "content-disposition",

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

91 "content-encoding", "content-language",

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

92 "content-md5", "content-type",

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

93 "x-robots-tag", "expires"])

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

94 _underscore_base_user_settable_fields = set()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

95 for f in base_user_settable_fields:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

96 _underscore_base_user_settable_fields.add(f.replace('-', '_'))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

97 # Metadata fields, whether user-settable or not, other than custom

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

98 # metadata fields (i.e., those beginning with a provider specific prefix

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

99 # like x-amz-meta).

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

100 base_fields = (base_user_settable_fields |

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

101 set(["last-modified", "content-length", "date", "etag"]))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

102

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

103

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

104

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

105 def __init__(self, bucket=None, name=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

106 self.bucket = bucket

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

107 self.name = name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

108 self.metadata = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

109 self.cache_control = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

110 self.content_type = self.DefaultContentType

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

111 self.content_encoding = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

112 self.content_disposition = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

113 self.content_language = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

114 self.filename = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

115 self.etag = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

116 self.is_latest = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

117 self.last_modified = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

118 self.owner = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

119 self._storage_class = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

120 self.path = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

121 self.resp = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

122 self.mode = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

123 self.size = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

124 self.version_id = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

125 self.source_version_id = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

126 self.delete_marker = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

127 self.encrypted = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

128 # If the object is being restored, this attribute will be set to True.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

129 # If the object is restored, it will be set to False. Otherwise this

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

130 # value will be None. If the restore is completed (ongoing_restore =

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

131 # False), the expiry_date will be populated with the expiry date of the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

132 # restored object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

133 self.ongoing_restore = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

134 self.expiry_date = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

135 self.local_hashes = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

136

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

137 def __repr__(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

138 if self.bucket:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

139 name = u'<Key: %s,%s>' % (self.bucket.name, self.name)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

140 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

141 name = u'<Key: None,%s>' % self.name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

142

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

143 # Encode to bytes for Python 2 to prevent display decoding issues

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

144 if not isinstance(name, str):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

145 name = name.encode('utf-8')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

146

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

147 return name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

148

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

149 def __iter__(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

150 return self

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

151

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

152 @property

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

153 def provider(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

154 provider = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

155 if self.bucket and self.bucket.connection:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

156 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

157 return provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

158

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

159 def _get_key(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

160 return self.name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

161

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

162 def _set_key(self, value):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

163 self.name = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

164

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

165 key = property(_get_key, _set_key);

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

166

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

167 def _get_md5(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

168 if 'md5' in self.local_hashes and self.local_hashes['md5']:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

169 return binascii.b2a_hex(self.local_hashes['md5'])

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

170

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

171 def _set_md5(self, value):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

172 if value:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

173 self.local_hashes['md5'] = binascii.a2b_hex(value)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

174 elif 'md5' in self.local_hashes:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

175 self.local_hashes.pop('md5', None)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

176

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

177 md5 = property(_get_md5, _set_md5);

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

178

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

179 def _get_base64md5(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

180 if 'md5' in self.local_hashes and self.local_hashes['md5']:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

181 md5 = self.local_hashes['md5']

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

182 if not isinstance(md5, bytes):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

183 md5 = md5.encode('utf-8')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

184 return binascii.b2a_base64(md5).decode('utf-8').rstrip('\n')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

185

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

186 def _set_base64md5(self, value):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

187 if value:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

188 if not isinstance(value, six.string_types):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

189 value = value.decode('utf-8')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

190 self.local_hashes['md5'] = binascii.a2b_base64(value)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

191 elif 'md5' in self.local_hashes:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

192 del self.local_hashes['md5']

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

193

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

194 base64md5 = property(_get_base64md5, _set_base64md5);

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

195

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

196 def _get_storage_class(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

197 if self._storage_class is None and self.bucket:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

198 # Attempt to fetch storage class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

199 list_items = list(self.bucket.list(self.name.encode('utf-8')))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

200 if len(list_items) and getattr(list_items[0], '_storage_class',

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

201 None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

202 self._storage_class = list_items[0]._storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

203 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

204 # Key is not yet saved? Just use default...

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

205 self._storage_class = 'STANDARD'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

206

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

207 return self._storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

208

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

209 def _set_storage_class(self, value):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

210 self._storage_class = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

211

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

212 storage_class = property(_get_storage_class, _set_storage_class)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

213

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

214 def get_md5_from_hexdigest(self, md5_hexdigest):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

215 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

216 A utility function to create the 2-tuple (md5hexdigest, base64md5)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

217 from just having a precalculated md5_hexdigest.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

218 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

219 digest = binascii.unhexlify(md5_hexdigest)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

220 base64md5 = encodebytes(digest)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

221 if base64md5[-1] == '\n':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

222 base64md5 = base64md5[0:-1]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

223 return (md5_hexdigest, base64md5)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

224

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

225 def handle_encryption_headers(self, resp):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

226 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

227 if provider.server_side_encryption_header:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

228 self.encrypted = resp.getheader(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

229 provider.server_side_encryption_header, None)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

230 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

231 self.encrypted = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

232

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

233 def handle_version_headers(self, resp, force=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

234 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

235 # If the Key object already has a version_id attribute value, it

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

236 # means that it represents an explicit version and the user is

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

237 # doing a get_contents_*(version_id=<foo>) to retrieve another

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

238 # version of the Key. In that case, we don't really want to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

239 # overwrite the version_id in this Key object. Comprende?

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

240 if self.version_id is None or force:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

241 self.version_id = resp.getheader(provider.version_id, None)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

242 self.source_version_id = resp.getheader(provider.copy_source_version_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

243 None)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

244 if resp.getheader(provider.delete_marker, 'false') == 'true':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

245 self.delete_marker = True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

246 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

247 self.delete_marker = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

248

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

249 def handle_restore_headers(self, response):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

250 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

251 header = response.getheader(provider.restore_header)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

252 if header is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

253 return

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

254 parts = header.split(',', 1)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

255 for part in parts:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

256 key, val = [i.strip() for i in part.split('=')]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

257 val = val.replace('"', '')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

258 if key == 'ongoing-request':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

259 self.ongoing_restore = True if val.lower() == 'true' else False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

260 elif key == 'expiry-date':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

261 self.expiry_date = val

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

262

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

263 def handle_addl_headers(self, headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

264 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

265 Used by Key subclasses to do additional, provider-specific

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

266 processing of response headers. No-op for this base class.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

267 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

268 pass

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

269

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

270 def open_read(self, headers=None, query_args='',

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

271 override_num_retries=None, response_headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

272 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

273 Open this key for reading

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

274

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

275 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

276 :param headers: Headers to pass in the web request

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

277

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

278 :type query_args: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

279 :param query_args: Arguments to pass in the query string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

280 (ie, 'torrent')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

281

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

282 :type override_num_retries: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

283 :param override_num_retries: If not None will override configured

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

284 num_retries parameter for underlying GET.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

285

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

286 :type response_headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

287 :param response_headers: A dictionary containing HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

288 headers/values that will override any headers associated

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

289 with the stored object in the response. See

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

290 http://goo.gl/EWOPb for details.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

291 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

292 if self.resp is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

293 self.mode = 'r'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

294

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

295 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

296 self.resp = self.bucket.connection.make_request(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

297 'GET', self.bucket.name, self.name, headers,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

298 query_args=query_args,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

299 override_num_retries=override_num_retries)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

300 if self.resp.status < 199 or self.resp.status > 299:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

301 body = self.resp.read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

302 raise provider.storage_response_error(self.resp.status,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

303 self.resp.reason, body)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

304 response_headers = self.resp.msg

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

305 self.metadata = boto.utils.get_aws_metadata(response_headers,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

306 provider)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

307 for name, value in response_headers.items():

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

308 # To get correct size for Range GETs, use Content-Range

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

309 # header if one was returned. If not, use Content-Length

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

310 # header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

311 if (name.lower() == 'content-length' and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

312 'Content-Range' not in response_headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

313 self.size = int(value)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

314 elif name.lower() == 'content-range':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

315 end_range = re.sub('.*/(.*)', '\\1', value)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

316 self.size = int(end_range)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

317 elif name.lower() in Key.base_fields:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

318 self.__dict__[name.lower().replace('-', '_')] = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

319 self.handle_version_headers(self.resp)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

320 self.handle_encryption_headers(self.resp)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

321 self.handle_restore_headers(self.resp)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

322 self.handle_addl_headers(self.resp.getheaders())

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

323

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

324 def open_write(self, headers=None, override_num_retries=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

325 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

326 Open this key for writing.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

327 Not yet implemented

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

328

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

329 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

330 :param headers: Headers to pass in the write request

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

331

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

332 :type override_num_retries: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

333 :param override_num_retries: If not None will override configured

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

334 num_retries parameter for underlying PUT.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

335 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

336 raise BotoClientError('Not Implemented')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

337

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

338 def open(self, mode='r', headers=None, query_args=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

339 override_num_retries=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

340 if mode == 'r':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

341 self.mode = 'r'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

342 self.open_read(headers=headers, query_args=query_args,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

343 override_num_retries=override_num_retries)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

344 elif mode == 'w':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

345 self.mode = 'w'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

346 self.open_write(headers=headers,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

347 override_num_retries=override_num_retries)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

348 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

349 raise BotoClientError('Invalid mode: %s' % mode)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

350

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

351 closed = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

352

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

353 def close(self, fast=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

354 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

355 Close this key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

356

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

357 :type fast: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

358 :param fast: True if you want the connection to be closed without first

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

359 reading the content. This should only be used in cases where subsequent

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

360 calls don't need to return the content from the open HTTP connection.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

361 Note: As explained at

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

362 http://docs.python.org/2/library/httplib.html#httplib.HTTPConnection.getresponse,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

363 callers must read the whole response before sending a new request to the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

364 server. Calling Key.close(fast=True) and making a subsequent request to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

365 the server will work because boto will get an httplib exception and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

366 close/reopen the connection.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

367

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

368 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

369 if self.resp and not fast:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

370 self.resp.read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

371 self.resp = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

372 self.mode = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

373 self.closed = True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

374

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

375 def next(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

376 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

377 By providing a next method, the key object supports use as an iterator.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

378 For example, you can now say:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

379

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

380 for bytes in key:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

381 write bytes to a file or whatever

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

382

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

383 All of the HTTP connection stuff is handled for you.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

384 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

385 self.open_read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

386 data = self.resp.read(self.BufferSize)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

387 if not data:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

388 self.close()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

389 raise StopIteration

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

390 return data

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

391

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

392 # Python 3 iterator support

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

393 __next__ = next

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

394

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

395 def read(self, size=0):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

396 self.open_read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

397 if size == 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

398 data = self.resp.read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

399 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

400 data = self.resp.read(size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

401 if not data:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

402 self.close()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

403 return data

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

404

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

405 def change_storage_class(self, new_storage_class, dst_bucket=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

406 validate_dst_bucket=True):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

407 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

408 Change the storage class of an existing key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

409 Depending on whether a different destination bucket is supplied

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

410 or not, this will either move the item within the bucket, preserving

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

411 all metadata and ACL info bucket changing the storage class or it

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

412 will copy the item to the provided destination bucket, also

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

413 preserving metadata and ACL info.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

414

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

415 :type new_storage_class: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

416 :param new_storage_class: The new storage class for the Key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

417 Possible values are:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

418 * STANDARD

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

419 * REDUCED_REDUNDANCY

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

420

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

421 :type dst_bucket: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

422 :param dst_bucket: The name of a destination bucket. If not

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

423 provided the current bucket of the key will be used.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

424

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

425 :type validate_dst_bucket: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

426 :param validate_dst_bucket: If True, will validate the dst_bucket

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

427 by using an extra list request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

428 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

429 bucket_name = dst_bucket or self.bucket.name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

430 if new_storage_class == 'STANDARD':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

431 return self.copy(bucket_name, self.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

432 reduced_redundancy=False, preserve_acl=True,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

433 validate_dst_bucket=validate_dst_bucket)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

434 elif new_storage_class == 'REDUCED_REDUNDANCY':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

435 return self.copy(bucket_name, self.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

436 reduced_redundancy=True, preserve_acl=True,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

437 validate_dst_bucket=validate_dst_bucket)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

438 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

439 raise BotoClientError('Invalid storage class: %s' %

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

440 new_storage_class)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

441

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

442 def copy(self, dst_bucket, dst_key, metadata=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

443 reduced_redundancy=False, preserve_acl=False,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

444 encrypt_key=False, validate_dst_bucket=True):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

445 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

446 Copy this Key to another bucket.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

447

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

448 :type dst_bucket: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

449 :param dst_bucket: The name of the destination bucket

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

450

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

451 :type dst_key: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

452 :param dst_key: The name of the destination key

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

453

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

454 :type metadata: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

455 :param metadata: Metadata to be associated with new key. If

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

456 metadata is supplied, it will replace the metadata of the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

457 source key being copied. If no metadata is supplied, the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

458 source key's metadata will be copied to the new key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

459

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

460 :type reduced_redundancy: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

461 :param reduced_redundancy: If True, this will force the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

462 storage class of the new Key to be REDUCED_REDUNDANCY

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

463 regardless of the storage class of the key being copied.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

464 The Reduced Redundancy Storage (RRS) feature of S3,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

465 provides lower redundancy at lower storage cost.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

466

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

467 :type preserve_acl: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

468 :param preserve_acl: If True, the ACL from the source key will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

469 be copied to the destination key. If False, the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

470 destination key will have the default ACL. Note that

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

471 preserving the ACL in the new key object will require two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

472 additional API calls to S3, one to retrieve the current

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

473 ACL and one to set that ACL on the new object. If you

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

474 don't care about the ACL, a value of False will be

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

475 significantly more efficient.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

476

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

477 :type encrypt_key: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

478 :param encrypt_key: If True, the new copy of the object will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

479 be encrypted on the server-side by S3 and will be stored

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

480 in an encrypted form while at rest in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

481

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

482 :type validate_dst_bucket: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

483 :param validate_dst_bucket: If True, will validate the dst_bucket

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

484 by using an extra list request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

485

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

486 :rtype: :class:`boto.s3.key.Key` or subclass

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

487 :returns: An instance of the newly created key object

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

488 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

489 dst_bucket = self.bucket.connection.lookup(dst_bucket,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

490 validate_dst_bucket)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

491 if reduced_redundancy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

492 storage_class = 'REDUCED_REDUNDANCY'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

493 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

494 storage_class = self.storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

495 return dst_bucket.copy_key(dst_key, self.bucket.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

496 self.name, metadata,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

497 storage_class=storage_class,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

498 preserve_acl=preserve_acl,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

499 encrypt_key=encrypt_key,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

500 src_version_id=self.version_id)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

501

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

502 def startElement(self, name, attrs, connection):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

503 if name == 'Owner':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

504 self.owner = User(self)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

505 return self.owner

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

506 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

507 return None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

508

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

509 def endElement(self, name, value, connection):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

510 if name == 'Key':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

511 self.name = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

512 elif name == 'ETag':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

513 self.etag = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

514 elif name == 'IsLatest':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

515 if value == 'true':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

516 self.is_latest = True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

517 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

518 self.is_latest = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

519 elif name == 'LastModified':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

520 self.last_modified = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

521 elif name == 'Size':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

522 self.size = int(value)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

523 elif name == 'StorageClass':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

524 self.storage_class = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

525 elif name == 'Owner':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

526 pass

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

527 elif name == 'VersionId':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

528 self.version_id = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

529 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

530 setattr(self, name, value)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

531

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

532 def exists(self, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

533 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

534 Returns True if the key exists

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

535

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

536 :rtype: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

537 :return: Whether the key exists on S3

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

538 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

539 return bool(self.bucket.lookup(self.name, headers=headers))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

540

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

541 def delete(self, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

542 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

543 Delete this key from S3

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

544 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

545 return self.bucket.delete_key(self.name, version_id=self.version_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

546 headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

547

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

548 def get_metadata(self, name):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

549 return self.metadata.get(name)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

550

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

551 def set_metadata(self, name, value):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

552 # Ensure that metadata that is vital to signing is in the correct

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

553 # case. Applies to ``Content-Type`` & ``Content-MD5``.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

554 if name.lower() == 'content-type':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

555 self.metadata['Content-Type'] = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

556 elif name.lower() == 'content-md5':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

557 self.metadata['Content-MD5'] = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

558 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

559 self.metadata[name] = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

560 if name.lower() in Key.base_user_settable_fields:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

561 self.__dict__[name.lower().replace('-', '_')] = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

562

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

563 def update_metadata(self, d):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

564 self.metadata.update(d)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

565

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

566 # convenience methods for setting/getting ACL

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

567 def set_acl(self, acl_str, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

568 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

569 self.bucket.set_acl(acl_str, self.name, headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

570

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

571 def get_acl(self, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

572 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

573 return self.bucket.get_acl(self.name, headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

574

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

575 def get_xml_acl(self, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

576 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

577 return self.bucket.get_xml_acl(self.name, headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

578

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

579 def set_xml_acl(self, acl_str, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

580 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

581 return self.bucket.set_xml_acl(acl_str, self.name, headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

582

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

583 def set_canned_acl(self, acl_str, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

584 return self.bucket.set_canned_acl(acl_str, self.name, headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

585

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

586 def get_redirect(self):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

587 """Return the redirect location configured for this key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

588

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

589 If no redirect is configured (via set_redirect), then None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

590 will be returned.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

591

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

592 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

593 response = self.bucket.connection.make_request(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

594 'HEAD', self.bucket.name, self.name)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

595 if response.status == 200:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

596 return response.getheader('x-amz-website-redirect-location')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

597 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

598 raise self.provider.storage_response_error(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

599 response.status, response.reason, response.read())

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

600

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

601 def set_redirect(self, redirect_location, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

602 """Configure this key to redirect to another location.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

603

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

604 When the bucket associated with this key is accessed from the website

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

605 endpoint, a 301 redirect will be issued to the specified

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

606 `redirect_location`.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

607

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

608 :type redirect_location: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

609 :param redirect_location: The location to redirect.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

610

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

611 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

612 if headers is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

613 headers = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

614 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

615 headers = headers.copy()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

616

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

617 headers['x-amz-website-redirect-location'] = redirect_location

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

618 response = self.bucket.connection.make_request('PUT', self.bucket.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

619 self.name, headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

620 if response.status == 200:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

621 return True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

622 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

623 raise self.provider.storage_response_error(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

624 response.status, response.reason, response.read())

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

625

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

626 def make_public(self, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

627 return self.bucket.set_canned_acl('public-read', self.name, headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

628

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

629 def generate_url(self, expires_in, method='GET', headers=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

630 query_auth=True, force_http=False, response_headers=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

631 expires_in_absolute=False, version_id=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

632 policy=None, reduced_redundancy=False, encrypt_key=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

633 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

634 Generate a URL to access this key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

635

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

636 :type expires_in: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

637 :param expires_in: How long the url is valid for, in seconds

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

638

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

639 :type method: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

640 :param method: The method to use for retrieving the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

641 (default is GET)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

642

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

643 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

644 :param headers: Any headers to pass along in the request

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

645

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

646 :type query_auth: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

647 :param query_auth:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

648

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

649 :type force_http: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

650 :param force_http: If True, http will be used instead of https.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

651

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

652 :type response_headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

653 :param response_headers: A dictionary containing HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

654 headers/values that will override any headers associated

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

655 with the stored object in the response. See

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

656 http://goo.gl/EWOPb for details.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

657

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

658 :type expires_in_absolute: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

659 :param expires_in_absolute:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

660

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

661 :type version_id: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

662 :param version_id: The version_id of the object to GET. If specified

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

663 this overrides any value in the key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

664

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

665 :type policy: :class:`boto.s3.acl.CannedACLStrings`

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

666 :param policy: A canned ACL policy that will be applied to the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

667 new key in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

668

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

669 :type reduced_redundancy: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

670 :param reduced_redundancy: If True, this will set the storage

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

671 class of the new Key to be REDUCED_REDUNDANCY. The Reduced

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

672 Redundancy Storage (RRS) feature of S3, provides lower

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

673 redundancy at lower storage cost.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

674

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

675 :type encrypt_key: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

676 :param encrypt_key: If True, the new copy of the object will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

677 be encrypted on the server-side by S3 and will be stored

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

678 in an encrypted form while at rest in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

679

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

680 :rtype: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

681 :return: The URL to access the key

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

682 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

683 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

684 version_id = version_id or self.version_id

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

685 if headers is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

686 headers = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

687 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

688 headers = headers.copy()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

689

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

690 # add headers accordingly (usually PUT case)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

691 if policy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

692 headers[provider.acl_header] = policy

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

693 if reduced_redundancy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

694 self.storage_class = 'REDUCED_REDUNDANCY'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

695 if provider.storage_class_header:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

696 headers[provider.storage_class_header] = self.storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

697 if encrypt_key:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

698 headers[provider.server_side_encryption_header] = 'AES256'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

699 headers = boto.utils.merge_meta(headers, self.metadata, provider)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

700

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

701 return self.bucket.connection.generate_url(expires_in, method,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

702 self.bucket.name, self.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

703 headers, query_auth,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

704 force_http,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

705 response_headers,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

706 expires_in_absolute,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

707 version_id)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

708

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

709 def send_file(self, fp, headers=None, cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

710 query_args=None, chunked_transfer=False, size=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

711 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

712 Upload a file to a key into a bucket on S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

713

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

714 :type fp: file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

715 :param fp: The file pointer to upload. The file pointer must

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

716 point point at the offset from which you wish to upload.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

717 ie. if uploading the full file, it should point at the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

718 start of the file. Normally when a file is opened for

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

719 reading, the fp will point at the first byte. See the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

720 bytes parameter below for more info.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

721

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

722 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

723 :param headers: The headers to pass along with the PUT request

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

724

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

725 :type num_cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

726 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

727 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

728 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

729 callback will be called during the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

730 transfer. Providing a negative integer will cause your

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

731 callback to be called with each buffer read.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

732

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

733 :type query_args: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

734 :param query_args: (optional) Arguments to pass in the query string.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

735

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

736 :type chunked_transfer: boolean

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

737 :param chunked_transfer: (optional) If true, we use chunked

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

738 Transfer-Encoding.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

739

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

740 :type size: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

741 :param size: (optional) The Maximum number of bytes to read

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

742 from the file pointer (fp). This is useful when uploading

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

743 a file in multiple parts where you are splitting the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

744 up into different ranges to be uploaded. If not specified,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

745 the default behaviour is to read all bytes from the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

746 pointer. Less bytes may be available.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

747 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

748 self._send_file_internal(fp, headers=headers, cb=cb, num_cb=num_cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

749 query_args=query_args,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

750 chunked_transfer=chunked_transfer, size=size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

751

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

752 def _send_file_internal(self, fp, headers=None, cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

753 query_args=None, chunked_transfer=False, size=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

754 hash_algs=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

755 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

756 try:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

757 spos = fp.tell()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

758 except IOError:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

759 spos = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

760 self.read_from_stream = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

761

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

762 # If hash_algs is unset and the MD5 hasn't already been computed,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

763 # default to an MD5 hash_alg to hash the data on-the-fly.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

764 if hash_algs is None and not self.md5:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

765 hash_algs = {'md5': md5}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

766 digesters = dict((alg, hash_algs[alg]()) for alg in hash_algs or {})

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

767

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

768 def sender(http_conn, method, path, data, headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

769 # This function is called repeatedly for temporary retries

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

770 # so we must be sure the file pointer is pointing at the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

771 # start of the data.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

772 if spos is not None and spos != fp.tell():

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

773 fp.seek(spos)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

774 elif spos is None and self.read_from_stream:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

775 # if seek is not supported, and we've read from this

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

776 # stream already, then we need to abort retries to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

777 # avoid setting bad data.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

778 raise provider.storage_data_error(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

779 'Cannot retry failed request. fp does not support seeking.')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

780

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

781 # If the caller explicitly specified host header, tell putrequest

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

782 # not to add a second host header. Similarly for accept-encoding.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

783 skips = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

784 if boto.utils.find_matching_headers('host', headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

785 skips['skip_host'] = 1

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

786 if boto.utils.find_matching_headers('accept-encoding', headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

787 skips['skip_accept_encoding'] = 1

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

788 http_conn.putrequest(method, path, **skips)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

789 for key in headers:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

790 http_conn.putheader(key, headers[key])

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

791 http_conn.endheaders()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

792

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

793 save_debug = self.bucket.connection.debug

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

794 self.bucket.connection.debug = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

795 # If the debuglevel < 4 we don't want to show connection

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

796 # payload, so turn off HTTP connection-level debug output (to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

797 # be restored below).

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

798 # Use the getattr approach to allow this to work in AppEngine.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

799 if getattr(http_conn, 'debuglevel', 0) < 4:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

800 http_conn.set_debuglevel(0)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

801

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

802 data_len = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

803 if cb:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

804 if size:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

805 cb_size = size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

806 elif self.size:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

807 cb_size = self.size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

808 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

809 cb_size = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

810 if chunked_transfer and cb_size == 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

811 # For chunked Transfer, we call the cb for every 1MB

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

812 # of data transferred, except when we know size.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

813 cb_count = (1024 * 1024) / self.BufferSize

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

814 elif num_cb > 1:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

815 cb_count = int(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

816 math.ceil(cb_size / self.BufferSize / (num_cb - 1.0)))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

817 elif num_cb < 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

818 cb_count = -1

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

819 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

820 cb_count = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

821 i = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

822 cb(data_len, cb_size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

823

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

824 bytes_togo = size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

825 if bytes_togo and bytes_togo < self.BufferSize:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

826 chunk = fp.read(bytes_togo)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

827 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

828 chunk = fp.read(self.BufferSize)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

829

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

830 if not isinstance(chunk, bytes):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

831 chunk = chunk.encode('utf-8')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

832

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

833 if spos is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

834 # read at least something from a non-seekable fp.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

835 self.read_from_stream = True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

836 while chunk:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

837 chunk_len = len(chunk)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

838 data_len += chunk_len

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

839 if chunked_transfer:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

840 http_conn.send('%x;\r\n' % chunk_len)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

841 http_conn.send(chunk)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

842 http_conn.send('\r\n')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

843 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

844 http_conn.send(chunk)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

845 for alg in digesters:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

846 digesters[alg].update(chunk)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

847 if bytes_togo:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

848 bytes_togo -= chunk_len

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

849 if bytes_togo <= 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

850 break

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

851 if cb:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

852 i += 1

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

853 if i == cb_count or cb_count == -1:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

854 cb(data_len, cb_size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

855 i = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

856 if bytes_togo and bytes_togo < self.BufferSize:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

857 chunk = fp.read(bytes_togo)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

858 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

859 chunk = fp.read(self.BufferSize)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

860

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

861 if not isinstance(chunk, bytes):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

862 chunk = chunk.encode('utf-8')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

863

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

864 self.size = data_len

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

865

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

866 for alg in digesters:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

867 self.local_hashes[alg] = digesters[alg].digest()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

868

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

869 if chunked_transfer:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

870 http_conn.send('0\r\n')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

871 # http_conn.send("Content-MD5: %s\r\n" % self.base64md5)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

872 http_conn.send('\r\n')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

873

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

874 if cb and (cb_count <= 1 or i > 0) and data_len > 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

875 cb(data_len, cb_size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

876

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

877 http_conn.set_debuglevel(save_debug)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

878 self.bucket.connection.debug = save_debug

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

879 response = http_conn.getresponse()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

880 body = response.read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

881

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

882 if not self.should_retry(response, chunked_transfer):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

883 raise provider.storage_response_error(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

884 response.status, response.reason, body)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

885

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

886 return response

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

887

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

888 if not headers:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

889 headers = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

890 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

891 headers = headers.copy()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

892 # Overwrite user-supplied user-agent.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

893 for header in find_matching_headers('User-Agent', headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

894 del headers[header]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

895 headers['User-Agent'] = UserAgent

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

896 # If storage_class is None, then a user has not explicitly requested

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

897 # a storage class, so we can assume STANDARD here

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

898 if self._storage_class not in [None, 'STANDARD']:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

899 headers[provider.storage_class_header] = self.storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

900 if find_matching_headers('Content-Encoding', headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

901 self.content_encoding = merge_headers_by_name(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

902 'Content-Encoding', headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

903 if find_matching_headers('Content-Language', headers):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

904 self.content_language = merge_headers_by_name(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

905 'Content-Language', headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

906 content_type_headers = find_matching_headers('Content-Type', headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

907 if content_type_headers:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

908 # Some use cases need to suppress sending of the Content-Type

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

909 # header and depend on the receiving server to set the content

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

910 # type. This can be achieved by setting headers['Content-Type']

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

911 # to None when calling this method.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

912 if (len(content_type_headers) == 1 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

913 headers[content_type_headers[0]] is None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

914 # Delete null Content-Type value to skip sending that header.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

915 del headers[content_type_headers[0]]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

916 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

917 self.content_type = merge_headers_by_name(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

918 'Content-Type', headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

919 elif self.path:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

920 self.content_type = mimetypes.guess_type(self.path)[0]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

921 if self.content_type is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

922 self.content_type = self.DefaultContentType

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

923 headers['Content-Type'] = self.content_type

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

924 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

925 headers['Content-Type'] = self.content_type

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

926 if self.base64md5:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

927 headers['Content-MD5'] = self.base64md5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

928 if chunked_transfer:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

929 headers['Transfer-Encoding'] = 'chunked'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

930 #if not self.base64md5:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

931 # headers['Trailer'] = "Content-MD5"

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

932 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

933 headers['Content-Length'] = str(self.size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

934 # This is terrible. We need a SHA256 of the body for SigV4, but to do

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

935 # the chunked ``sender`` behavior above, the ``fp`` isn't available to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

936 # the auth mechanism (because closures). Detect if it's SigV4 & embelish

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

937 # while we can before the auth calculations occur.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

938 if 'hmac-v4-s3' in self.bucket.connection._required_auth_capability():

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

939 kwargs = {'fp': fp, 'hash_algorithm': hashlib.sha256}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

940 if size is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

941 kwargs['size'] = size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

942 headers['_sha256'] = compute_hash(**kwargs)[0]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

943 headers['Expect'] = '100-Continue'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

944 headers = boto.utils.merge_meta(headers, self.metadata, provider)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

945 resp = self.bucket.connection.make_request(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

946 'PUT',

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

947 self.bucket.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

948 self.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

949 headers,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

950 sender=sender,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

951 query_args=query_args

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

952 )

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

953 self.handle_version_headers(resp, force=True)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

954 self.handle_addl_headers(resp.getheaders())

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

955

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

956 def should_retry(self, response, chunked_transfer=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

957 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

958

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

959 if not chunked_transfer:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

960 if response.status in [500, 503]:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

961 # 500 & 503 can be plain retries.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

962 return True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

963

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

964 if response.getheader('location'):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

965 # If there's a redirect, plain retry.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

966 return True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

967

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

968 if 200 <= response.status <= 299:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

969 self.etag = response.getheader('etag')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

970 md5 = self.md5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

971 if isinstance(md5, bytes):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

972 md5 = md5.decode('utf-8')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

973

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

974 # If you use customer-provided encryption keys, the ETag value that

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

975 # Amazon S3 returns in the response will not be the MD5 of the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

976 # object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

977 server_side_encryption_customer_algorithm = response.getheader(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

978 'x-amz-server-side-encryption-customer-algorithm', None)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

979 if server_side_encryption_customer_algorithm is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

980 if self.etag != '"%s"' % md5:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

981 raise provider.storage_data_error(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

982 'ETag from S3 did not match computed MD5. '

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

983 '%s vs. %s' % (self.etag, self.md5))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

984

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

985 return True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

986

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

987 if response.status == 400:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

988 # The 400 must be trapped so the retry handler can check to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

989 # see if it was a timeout.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

990 # If ``RequestTimeout`` is present, we'll retry. Otherwise, bomb

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

991 # out.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

992 body = response.read()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

993 err = provider.storage_response_error(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

994 response.status,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

995 response.reason,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

996 body

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

997 )

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

998

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

999 if err.error_code in ['RequestTimeout']:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1000 raise PleaseRetryException(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1001 "Saw %s, retrying" % err.error_code,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1002 response=response

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1003 )

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1004

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1005 return False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1006

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1007 def compute_md5(self, fp, size=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1008 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1009 :type fp: file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1010 :param fp: File pointer to the file to MD5 hash. The file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1011 pointer will be reset to the same position before the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1012 method returns.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1013

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1014 :type size: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1015 :param size: (optional) The Maximum number of bytes to read

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1016 from the file pointer (fp). This is useful when uploading

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1017 a file in multiple parts where the file is being split

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1018 in place into different parts. Less bytes may be available.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1019 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1020 hex_digest, b64_digest, data_size = compute_md5(fp, size=size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1021 # Returned values are MD5 hash, base64 encoded MD5 hash, and data size.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1022 # The internal implementation of compute_md5() needs to return the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1023 # data size but we don't want to return that value to the external

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1024 # caller because it changes the class interface (i.e. it might

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1025 # break some code) so we consume the third tuple value here and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1026 # return the remainder of the tuple to the caller, thereby preserving

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1027 # the existing interface.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1028 self.size = data_size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1029 return (hex_digest, b64_digest)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1030

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1031 def set_contents_from_stream(self, fp, headers=None, replace=True,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1032 cb=None, num_cb=10, policy=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1033 reduced_redundancy=False, query_args=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1034 size=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1035 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1036 Store an object using the name of the Key object as the key in

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1037 cloud and the contents of the data stream pointed to by 'fp' as

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1038 the contents.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1039

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1040 The stream object is not seekable and total size is not known.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1041 This has the implication that we can't specify the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1042 Content-Size and Content-MD5 in the header. So for huge

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1043 uploads, the delay in calculating MD5 is avoided but with a

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1044 penalty of inability to verify the integrity of the uploaded

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1045 data.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1046

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1047 :type fp: file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1048 :param fp: the file whose contents are to be uploaded

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1049

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1050 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1051 :param headers: additional HTTP headers to be sent with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1052 PUT request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1053

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1054 :type replace: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1055 :param replace: If this parameter is False, the method will first check

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1056 to see if an object exists in the bucket with the same key. If it

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1057 does, it won't overwrite it. The default value is True which will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1058 overwrite the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1059

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1060 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1061 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1062 progress on the upload. The callback should accept two integer

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1063 parameters, the first representing the number of bytes that have

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1064 been successfully transmitted to GS and the second representing the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1065 total number of bytes that need to be transmitted.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1066

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1067 :type num_cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1068 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1069 cb parameter, this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1070 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1071 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1072

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1073 :type policy: :class:`boto.gs.acl.CannedACLStrings`

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1074 :param policy: A canned ACL policy that will be applied to the new key

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1075 in GS.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1076

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1077 :type reduced_redundancy: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1078 :param reduced_redundancy: If True, this will set the storage

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1079 class of the new Key to be REDUCED_REDUNDANCY. The Reduced

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1080 Redundancy Storage (RRS) feature of S3, provides lower

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1081 redundancy at lower storage cost.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1082

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1083 :type size: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1084 :param size: (optional) The Maximum number of bytes to read from

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1085 the file pointer (fp). This is useful when uploading a

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1086 file in multiple parts where you are splitting the file up

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1087 into different ranges to be uploaded. If not specified,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1088 the default behaviour is to read all bytes from the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1089 pointer. Less bytes may be available.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1090 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1091

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1092 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1093 if not provider.supports_chunked_transfer():

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1094 raise BotoClientError('%s does not support chunked transfer'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1095 % provider.get_provider_name())

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1096

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1097 # Name of the Object should be specified explicitly for Streams.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1098 if not self.name or self.name == '':

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1099 raise BotoClientError('Cannot determine the destination '

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1100 'object name for the given stream')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1101

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1102 if headers is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1103 headers = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1104 if policy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1105 headers[provider.acl_header] = policy

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1106

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1107 if reduced_redundancy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1108 self.storage_class = 'REDUCED_REDUNDANCY'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1109 if provider.storage_class_header:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1110 headers[provider.storage_class_header] = self.storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1111

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1112 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1113 if not replace:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1114 if self.bucket.lookup(self.name):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1115 return

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1116 self.send_file(fp, headers, cb, num_cb, query_args,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1117 chunked_transfer=True, size=size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1118

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1119 def set_contents_from_file(self, fp, headers=None, replace=True,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1120 cb=None, num_cb=10, policy=None, md5=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1121 reduced_redundancy=False, query_args=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1122 encrypt_key=False, size=None, rewind=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1123 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1124 Store an object in S3 using the name of the Key object as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1125 key in S3 and the contents of the file pointed to by 'fp' as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1126 contents. The data is read from 'fp' from its current position until

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1127 'size' bytes have been read or EOF.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1128

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1129 :type fp: file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1130 :param fp: the file whose contents to upload

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1131

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1132 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1133 :param headers: Additional HTTP headers that will be sent with

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1134 the PUT request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1135

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1136 :type replace: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1137 :param replace: If this parameter is False, the method will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1138 first check to see if an object exists in the bucket with

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1139 the same key. If it does, it won't overwrite it. The

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1140 default value is True which will overwrite the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1141

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1142 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1143 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1144 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1145 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1146 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1147 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1148 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1149

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1150 :type num_cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1151 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1152 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1153 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1154 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1155

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1156 :type policy: :class:`boto.s3.acl.CannedACLStrings`

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1157 :param policy: A canned ACL policy that will be applied to the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1158 new key in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1159

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1160 :type md5: A tuple containing the hexdigest version of the MD5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1161 checksum of the file as the first element and the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1162 Base64-encoded version of the plain checksum as the second

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1163 element. This is the same format returned by the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1164 compute_md5 method.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1165 :param md5: If you need to compute the MD5 for any reason

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1166 prior to upload, it's silly to have to do it twice so this

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1167 param, if present, will be used as the MD5 values of the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1168 file. Otherwise, the checksum will be computed.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1169

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1170 :type reduced_redundancy: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1171 :param reduced_redundancy: If True, this will set the storage

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1172 class of the new Key to be REDUCED_REDUNDANCY. The Reduced

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1173 Redundancy Storage (RRS) feature of S3, provides lower

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1174 redundancy at lower storage cost.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1175

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1176 :type encrypt_key: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1177 :param encrypt_key: If True, the new copy of the object will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1178 be encrypted on the server-side by S3 and will be stored

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1179 in an encrypted form while at rest in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1180

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1181 :type size: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1182 :param size: (optional) The Maximum number of bytes to read

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1183 from the file pointer (fp). This is useful when uploading

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1184 a file in multiple parts where you are splitting the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1185 up into different ranges to be uploaded. If not specified,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1186 the default behaviour is to read all bytes from the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1187 pointer. Less bytes may be available.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1188

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1189 :type rewind: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1190 :param rewind: (optional) If True, the file pointer (fp) will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1191 be rewound to the start before any bytes are read from

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1192 it. The default behaviour is False which reads from the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1193 current position of the file pointer (fp).

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1194

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1195 :rtype: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1196 :return: The number of bytes written to the key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1197 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1198 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1199 headers = headers or {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1200 if policy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1201 headers[provider.acl_header] = policy

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1202 if encrypt_key:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1203 headers[provider.server_side_encryption_header] = 'AES256'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1204

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1205 if rewind:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1206 # caller requests reading from beginning of fp.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1207 fp.seek(0, os.SEEK_SET)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1208 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1209 # The following seek/tell/seek logic is intended

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1210 # to detect applications using the older interface to

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1211 # set_contents_from_file(), which automatically rewound the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1212 # file each time the Key was reused. This changed with commit

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1213 # 14ee2d03f4665fe20d19a85286f78d39d924237e, to support uploads

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1214 # split into multiple parts and uploaded in parallel, and at

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1215 # the time of that commit this check was added because otherwise

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1216 # older programs would get a success status and upload an empty

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1217 # object. Unfortuantely, it's very inefficient for fp's implemented

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1218 # by KeyFile (used, for example, by gsutil when copying between

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1219 # providers). So, we skip the check for the KeyFile case.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1220 # TODO: At some point consider removing this seek/tell/seek

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1221 # logic, after enough time has passed that it's unlikely any

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1222 # programs remain that assume the older auto-rewind interface.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1223 if not isinstance(fp, KeyFile):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1224 spos = fp.tell()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1225 fp.seek(0, os.SEEK_END)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1226 if fp.tell() == spos:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1227 fp.seek(0, os.SEEK_SET)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1228 if fp.tell() != spos:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1229 # Raise an exception as this is likely a programming

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1230 # error whereby there is data before the fp but nothing

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1231 # after it.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1232 fp.seek(spos)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1233 raise AttributeError('fp is at EOF. Use rewind option '

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1234 'or seek() to data start.')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1235 # seek back to the correct position.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1236 fp.seek(spos)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1237

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1238 if reduced_redundancy:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1239 self.storage_class = 'REDUCED_REDUNDANCY'

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1240 if provider.storage_class_header:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1241 headers[provider.storage_class_header] = self.storage_class

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1242 # TODO - What if provider doesn't support reduced reduncancy?

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1243 # What if different providers provide different classes?

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1244 if hasattr(fp, 'name'):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1245 self.path = fp.name

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1246 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1247 if not md5 and provider.supports_chunked_transfer():

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1248 # defer md5 calculation to on the fly and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1249 # we don't know anything about size yet.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1250 chunked_transfer = True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1251 self.size = None

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1252 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1253 chunked_transfer = False

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1254 if isinstance(fp, KeyFile):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1255 # Avoid EOF seek for KeyFile case as it's very inefficient.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1256 key = fp.getkey()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1257 size = key.size - fp.tell()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1258 self.size = size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1259 # At present both GCS and S3 use MD5 for the etag for

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1260 # non-multipart-uploaded objects. If the etag is 32 hex

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1261 # chars use it as an MD5, to avoid having to read the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1262 # twice while transferring.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1263 if (re.match('^"[a-fA-F0-9]{32}"$', key.etag)):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1264 etag = key.etag.strip('"')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1265 md5 = (etag, base64.b64encode(binascii.unhexlify(etag)))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1266 if not md5:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1267 # compute_md5() and also set self.size to actual

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1268 # size of the bytes read computing the md5.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1269 md5 = self.compute_md5(fp, size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1270 # adjust size if required

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1271 size = self.size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1272 elif size:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1273 self.size = size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1274 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1275 # If md5 is provided, still need to size so

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1276 # calculate based on bytes to end of content

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1277 spos = fp.tell()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1278 fp.seek(0, os.SEEK_END)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1279 self.size = fp.tell() - spos

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1280 fp.seek(spos)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1281 size = self.size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1282 self.md5 = md5[0]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1283 self.base64md5 = md5[1]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1284

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1285 if self.name is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1286 self.name = self.md5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1287 if not replace:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1288 if self.bucket.lookup(self.name):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1289 return

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1290

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1291 self.send_file(fp, headers=headers, cb=cb, num_cb=num_cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1292 query_args=query_args,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1293 chunked_transfer=chunked_transfer, size=size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1294 # return number of bytes written.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1295 return self.size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1296

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1297 def set_contents_from_filename(self, filename, headers=None, replace=True,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1298 cb=None, num_cb=10, policy=None, md5=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1299 reduced_redundancy=False,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1300 encrypt_key=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1301 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1302 Store an object in S3 using the name of the Key object as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1303 key in S3 and the contents of the file named by 'filename'.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1304 See set_contents_from_file method for details about the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1305 parameters.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1306

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1307 :type filename: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1308 :param filename: The name of the file that you want to put onto S3

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1309

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1310 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1311 :param headers: Additional headers to pass along with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1312 request to AWS.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1313

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1314 :type replace: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1315 :param replace: If True, replaces the contents of the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1316 if it already exists.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1317

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1318 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1319 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1320 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1321 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1322 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1323 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1324 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1325

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1326 :type cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1327 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1328 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1329 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1330 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1331

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1332 :type policy: :class:`boto.s3.acl.CannedACLStrings`

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1333 :param policy: A canned ACL policy that will be applied to the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1334 new key in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1335

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1336 :type md5: A tuple containing the hexdigest version of the MD5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1337 checksum of the file as the first element and the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1338 Base64-encoded version of the plain checksum as the second

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1339 element. This is the same format returned by the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1340 compute_md5 method.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1341 :param md5: If you need to compute the MD5 for any reason

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1342 prior to upload, it's silly to have to do it twice so this

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1343 param, if present, will be used as the MD5 values of the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1344 file. Otherwise, the checksum will be computed.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1345

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1346 :type reduced_redundancy: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1347 :param reduced_redundancy: If True, this will set the storage

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1348 class of the new Key to be REDUCED_REDUNDANCY. The Reduced

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1349 Redundancy Storage (RRS) feature of S3, provides lower

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1350 redundancy at lower storage cost. :type encrypt_key: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1351 :param encrypt_key: If True, the new copy of the object

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1352 will be encrypted on the server-side by S3 and will be

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1353 stored in an encrypted form while at rest in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1354

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1355 :rtype: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1356 :return: The number of bytes written to the key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1357 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1358 with open(filename, 'rb') as fp:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1359 return self.set_contents_from_file(fp, headers, replace, cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1360 num_cb, policy, md5,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1361 reduced_redundancy,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1362 encrypt_key=encrypt_key)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1363

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1364 def set_contents_from_string(self, string_data, headers=None, replace=True,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1365 cb=None, num_cb=10, policy=None, md5=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1366 reduced_redundancy=False,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1367 encrypt_key=False):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1368 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1369 Store an object in S3 using the name of the Key object as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1370 key in S3 and the string 's' as the contents.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1371 See set_contents_from_file method for details about the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1372 parameters.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1373

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1374 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1375 :param headers: Additional headers to pass along with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1376 request to AWS.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1377

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1378 :type replace: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1379 :param replace: If True, replaces the contents of the file if

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1380 it already exists.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1381

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1382 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1383 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1384 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1385 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1386 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1387 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1388 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1389

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1390 :type cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1391 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1392 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1393 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1394 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1395

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1396 :type policy: :class:`boto.s3.acl.CannedACLStrings`

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1397 :param policy: A canned ACL policy that will be applied to the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1398 new key in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1399

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1400 :type md5: A tuple containing the hexdigest version of the MD5

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1401 checksum of the file as the first element and the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1402 Base64-encoded version of the plain checksum as the second

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1403 element. This is the same format returned by the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1404 compute_md5 method.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1405 :param md5: If you need to compute the MD5 for any reason

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1406 prior to upload, it's silly to have to do it twice so this

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1407 param, if present, will be used as the MD5 values of the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1408 file. Otherwise, the checksum will be computed.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1409

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1410 :type reduced_redundancy: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1411 :param reduced_redundancy: If True, this will set the storage

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1412 class of the new Key to be REDUCED_REDUNDANCY. The Reduced

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1413 Redundancy Storage (RRS) feature of S3, provides lower

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1414 redundancy at lower storage cost.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1415

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1416 :type encrypt_key: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1417 :param encrypt_key: If True, the new copy of the object will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1418 be encrypted on the server-side by S3 and will be stored

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1419 in an encrypted form while at rest in S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1420 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1421 if not isinstance(string_data, bytes):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1422 string_data = string_data.encode("utf-8")

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1423 fp = BytesIO(string_data)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1424 r = self.set_contents_from_file(fp, headers, replace, cb, num_cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1425 policy, md5, reduced_redundancy,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1426 encrypt_key=encrypt_key)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1427 fp.close()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1428 return r

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1429

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1430 def get_file(self, fp, headers=None, cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1431 torrent=False, version_id=None, override_num_retries=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1432 response_headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1433 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1434 Retrieves a file from an S3 Key

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1435

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1436 :type fp: file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1437 :param fp: File pointer to put the data into

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1438

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1439 :type headers: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1440 :param: headers to send when retrieving the files

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1441

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1442 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1443 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1444 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1445 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1446 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1447 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1448 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1449

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1450 :type cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1451 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1452 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1453 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1454 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1455

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1456 :type torrent: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1457 :param torrent: Flag for whether to get a torrent for the file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1458

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1459 :type override_num_retries: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1460 :param override_num_retries: If not None will override configured

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1461 num_retries parameter for underlying GET.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1462

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1463 :type response_headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1464 :param response_headers: A dictionary containing HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1465 headers/values that will override any headers associated

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1466 with the stored object in the response. See

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1467 http://goo.gl/EWOPb for details.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1468

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1469 :type version_id: str

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1470 :param version_id: The ID of a particular version of the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1471 If this parameter is not supplied but the Key object has

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1472 a ``version_id`` attribute, that value will be used when

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1473 retrieving the object. You can set the Key object's

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1474 ``version_id`` attribute to None to always grab the latest

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1475 version from a version-enabled bucket.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1476 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1477 self._get_file_internal(fp, headers=headers, cb=cb, num_cb=num_cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1478 torrent=torrent, version_id=version_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1479 override_num_retries=override_num_retries,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1480 response_headers=response_headers,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1481 hash_algs=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1482 query_args=None)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1483

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1484 def _get_file_internal(self, fp, headers=None, cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1485 torrent=False, version_id=None, override_num_retries=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1486 response_headers=None, hash_algs=None, query_args=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1487 if headers is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1488 headers = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1489 save_debug = self.bucket.connection.debug

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1490 if self.bucket.connection.debug == 1:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1491 self.bucket.connection.debug = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1492

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1493 query_args = query_args or []

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1494 if torrent:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1495 query_args.append('torrent')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1496

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1497 if hash_algs is None and not torrent:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1498 hash_algs = {'md5': md5}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1499 digesters = dict((alg, hash_algs[alg]()) for alg in hash_algs or {})

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1500

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1501 # If a version_id is passed in, use that. If not, check to see

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1502 # if the Key object has an explicit version_id and, if so, use that.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1503 # Otherwise, don't pass a version_id query param.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1504 if version_id is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1505 version_id = self.version_id

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1506 if version_id:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1507 query_args.append('versionId=%s' % version_id)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1508 if response_headers:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1509 for key in response_headers:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1510 query_args.append('%s=%s' % (

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1511 key, urllib.parse.quote(response_headers[key])))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1512 query_args = '&'.join(query_args)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1513 self.open('r', headers, query_args=query_args,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1514 override_num_retries=override_num_retries)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1515

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1516 data_len = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1517 if cb:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1518 if self.size is None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1519 cb_size = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1520 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1521 cb_size = self.size

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1522 if self.size is None and num_cb != -1:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1523 # If size is not available due to chunked transfer for example,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1524 # we'll call the cb for every 1MB of data transferred.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1525 cb_count = (1024 * 1024) / self.BufferSize

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1526 elif num_cb > 1:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1527 cb_count = int(math.ceil(cb_size/self.BufferSize/(num_cb-1.0)))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1528 elif num_cb < 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1529 cb_count = -1

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1530 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1531 cb_count = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1532 i = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1533 cb(data_len, cb_size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1534 try:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1535 for bytes in self:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1536 fp.write(bytes)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1537 data_len += len(bytes)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1538 for alg in digesters:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1539 digesters[alg].update(bytes)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1540 if cb:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1541 if cb_size > 0 and data_len >= cb_size:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1542 break

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1543 i += 1

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1544 if i == cb_count or cb_count == -1:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1545 cb(data_len, cb_size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1546 i = 0

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1547 except IOError as e:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1548 if e.errno == errno.ENOSPC:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1549 raise StorageDataError('Out of space for destination file '

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1550 '%s' % fp.name)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1551 raise

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1552 if cb and (cb_count <= 1 or i > 0) and data_len > 0:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1553 cb(data_len, cb_size)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1554 for alg in digesters:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1555 self.local_hashes[alg] = digesters[alg].digest()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1556 if self.size is None and not torrent and "Range" not in headers:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1557 self.size = data_len

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1558 self.close()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1559 self.bucket.connection.debug = save_debug

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1560

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1561 def get_torrent_file(self, fp, headers=None, cb=None, num_cb=10):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1562 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1563 Get a torrent file (see to get_file)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1564

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1565 :type fp: file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1566 :param fp: The file pointer of where to put the torrent

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1567

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1568 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1569 :param headers: Headers to be passed

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1570

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1571 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1572 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1573 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1574 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1575 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1576 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1577 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1578

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1579 :type cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1580 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1581 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1582 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1583 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1584

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1585 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1586 return self.get_file(fp, headers, cb, num_cb, torrent=True)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1587

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1588 def get_contents_to_file(self, fp, headers=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1589 cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1590 torrent=False,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1591 version_id=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1592 res_download_handler=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1593 response_headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1594 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1595 Retrieve an object from S3 using the name of the Key object as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1596 key in S3. Write the contents of the object to the file pointed

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1597 to by 'fp'.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1598

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1599 :type fp: File -like object

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1600 :param fp:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1601

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1602 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1603 :param headers: additional HTTP headers that will be sent with

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1604 the GET request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1605

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1606 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1607 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1608 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1609 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1610 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1611 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1612 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1613

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1614 :type cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1615 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1616 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1617 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1618 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1619

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1620 :type torrent: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1621 :param torrent: If True, returns the contents of a torrent

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1622 file as a string.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1623

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1624 :type res_upload_handler: ResumableDownloadHandler

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1625 :param res_download_handler: If provided, this handler will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1626 perform the download.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1627

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1628 :type response_headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1629 :param response_headers: A dictionary containing HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1630 headers/values that will override any headers associated

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1631 with the stored object in the response. See

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1632 http://goo.gl/EWOPb for details.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1633

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1634 :type version_id: str

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1635 :param version_id: The ID of a particular version of the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1636 If this parameter is not supplied but the Key object has

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1637 a ``version_id`` attribute, that value will be used when

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1638 retrieving the object. You can set the Key object's

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1639 ``version_id`` attribute to None to always grab the latest

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1640 version from a version-enabled bucket.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1641 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1642 if self.bucket is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1643 if res_download_handler:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1644 res_download_handler.get_file(self, fp, headers, cb, num_cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1645 torrent=torrent,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1646 version_id=version_id)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1647 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1648 self.get_file(fp, headers, cb, num_cb, torrent=torrent,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1649 version_id=version_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1650 response_headers=response_headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1651

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1652 def get_contents_to_filename(self, filename, headers=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1653 cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1654 torrent=False,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1655 version_id=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1656 res_download_handler=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1657 response_headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1658 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1659 Retrieve an object from S3 using the name of the Key object as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1660 key in S3. Store contents of the object to a file named by 'filename'.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1661 See get_contents_to_file method for details about the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1662 parameters.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1663

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1664 :type filename: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1665 :param filename: The filename of where to put the file contents

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1666

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1667 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1668 :param headers: Any additional headers to send in the request

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1669

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1670 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1671 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1672 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1673 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1674 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1675 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1676 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1677

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1678 :type num_cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1679 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1680 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1681 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1682 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1683

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1684 :type torrent: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1685 :param torrent: If True, returns the contents of a torrent file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1686 as a string.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1687

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1688 :type res_upload_handler: ResumableDownloadHandler

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1689 :param res_download_handler: If provided, this handler will

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1690 perform the download.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1691

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1692 :type response_headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1693 :param response_headers: A dictionary containing HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1694 headers/values that will override any headers associated

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1695 with the stored object in the response. See

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1696 http://goo.gl/EWOPb for details.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1697

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1698 :type version_id: str

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1699 :param version_id: The ID of a particular version of the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1700 If this parameter is not supplied but the Key object has

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1701 a ``version_id`` attribute, that value will be used when

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1702 retrieving the object. You can set the Key object's

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1703 ``version_id`` attribute to None to always grab the latest

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1704 version from a version-enabled bucket.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1705 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1706 try:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1707 with open(filename, 'wb') as fp:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1708 self.get_contents_to_file(fp, headers, cb, num_cb,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1709 torrent=torrent,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1710 version_id=version_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1711 res_download_handler=res_download_handler,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1712 response_headers=response_headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1713 except Exception:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1714 os.remove(filename)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1715 raise

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1716 # if last_modified date was sent from s3, try to set file's timestamp

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1717 if self.last_modified is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1718 try:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1719 modified_tuple = email.utils.parsedate_tz(self.last_modified)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1720 modified_stamp = int(email.utils.mktime_tz(modified_tuple))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1721 os.utime(fp.name, (modified_stamp, modified_stamp))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1722 except Exception:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1723 pass

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1724

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1725 def get_contents_as_string(self, headers=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1726 cb=None, num_cb=10,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1727 torrent=False,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1728 version_id=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1729 response_headers=None, encoding=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1730 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1731 Retrieve an object from S3 using the name of the Key object as the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1732 key in S3. Return the contents of the object as a string.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1733 See get_contents_to_file method for details about the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1734 parameters.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1735

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1736 :type headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1737 :param headers: Any additional headers to send in the request

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1738

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1739 :type cb: function

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1740 :param cb: a callback function that will be called to report

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1741 progress on the upload. The callback should accept two

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1742 integer parameters, the first representing the number of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1743 bytes that have been successfully transmitted to S3 and

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1744 the second representing the size of the to be transmitted

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1745 object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1746

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1747 :type cb: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1748 :param num_cb: (optional) If a callback is specified with the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1749 cb parameter this parameter determines the granularity of

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1750 the callback by defining the maximum number of times the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1751 callback will be called during the file transfer.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1752

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1753 :type torrent: bool

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1754 :param torrent: If True, returns the contents of a torrent file

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1755 as a string.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1756

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1757 :type response_headers: dict

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1758 :param response_headers: A dictionary containing HTTP

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1759 headers/values that will override any headers associated

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1760 with the stored object in the response. See

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1761 http://goo.gl/EWOPb for details.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1762

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1763 :type version_id: str

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1764 :param version_id: The ID of a particular version of the object.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1765 If this parameter is not supplied but the Key object has

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1766 a ``version_id`` attribute, that value will be used when

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1767 retrieving the object. You can set the Key object's

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1768 ``version_id`` attribute to None to always grab the latest

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1769 version from a version-enabled bucket.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1770

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1771 :type encoding: str

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1772 :param encoding: The text encoding to use, such as ``utf-8``

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1773 or ``iso-8859-1``. If set, then a string will be returned.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1774 Defaults to ``None`` and returns bytes.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1775

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1776 :rtype: bytes or str

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1777 :returns: The contents of the file as bytes or a string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1778 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1779 fp = BytesIO()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1780 self.get_contents_to_file(fp, headers, cb, num_cb, torrent=torrent,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1781 version_id=version_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1782 response_headers=response_headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1783 value = fp.getvalue()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1784

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1785 if encoding is not None:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1786 value = value.decode(encoding)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1787

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1788 return value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1789

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1790 def add_email_grant(self, permission, email_address, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1791 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1792 Convenience method that provides a quick way to add an email grant

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1793 to a key. This method retrieves the current ACL, creates a new

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1794 grant based on the parameters passed in, adds that grant to the ACL

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1795 and then PUT's the new ACL back to S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1796

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1797 :type permission: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1798 :param permission: The permission being granted. Should be one of:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1799 (READ, WRITE, READ_ACP, WRITE_ACP, FULL_CONTROL).

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1800

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1801 :type email_address: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1802 :param email_address: The email address associated with the AWS

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1803 account your are granting the permission to.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1804

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1805 :type recursive: boolean

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1806 :param recursive: A boolean value to controls whether the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1807 command will apply the grant to all keys within the bucket

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1808 or not. The default value is False. By passing a True

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1809 value, the call will iterate through all keys in the

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1810 bucket and apply the same grant to each key. CAUTION: If

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1811 you have a lot of keys, this could take a long time!

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1812 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1813 policy = self.get_acl(headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1814 policy.acl.add_email_grant(permission, email_address)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1815 self.set_acl(policy, headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1816

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1817 def add_user_grant(self, permission, user_id, headers=None,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1818 display_name=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1819 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1820 Convenience method that provides a quick way to add a canonical

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1821 user grant to a key. This method retrieves the current ACL,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1822 creates a new grant based on the parameters passed in, adds that

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1823 grant to the ACL and then PUT's the new ACL back to S3.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1824

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1825 :type permission: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1826 :param permission: The permission being granted. Should be one of:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1827 (READ, WRITE, READ_ACP, WRITE_ACP, FULL_CONTROL).

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1828

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1829 :type user_id: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1830 :param user_id: The canonical user id associated with the AWS

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1831 account your are granting the permission to.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1832

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1833 :type display_name: string

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1834 :param display_name: An option string containing the user's

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1835 Display Name. Only required on Walrus.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1836 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1837 policy = self.get_acl(headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1838 policy.acl.add_user_grant(permission, user_id,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1839 display_name=display_name)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1840 self.set_acl(policy, headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1841

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1842 def _normalize_metadata(self, metadata):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1843 if type(metadata) == set:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1844 norm_metadata = set()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1845 for k in metadata:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1846 norm_metadata.add(k.lower())

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1847 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1848 norm_metadata = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1849 for k in metadata:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1850 norm_metadata[k.lower()] = metadata[k]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1851 return norm_metadata

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1852

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1853 def _get_remote_metadata(self, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1854 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1855 Extracts metadata from existing URI into a dict, so we can

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1856 overwrite/delete from it to form the new set of metadata to apply to a

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1857 key.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1858 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1859 metadata = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1860 for underscore_name in self._underscore_base_user_settable_fields:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1861 if hasattr(self, underscore_name):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1862 value = getattr(self, underscore_name)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1863 if value:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1864 # Generate HTTP field name corresponding to "_" named field.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1865 field_name = underscore_name.replace('_', '-')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1866 metadata[field_name.lower()] = value

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1867 # self.metadata contains custom metadata, which are all user-settable.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1868 prefix = self.provider.metadata_prefix

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1869 for underscore_name in self.metadata:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1870 field_name = underscore_name.replace('_', '-')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1871 metadata['%s%s' % (prefix, field_name.lower())] = (

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1872 self.metadata[underscore_name])

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1873 return metadata

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1874

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1875 def set_remote_metadata(self, metadata_plus, metadata_minus, preserve_acl,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1876 headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1877 metadata_plus = self._normalize_metadata(metadata_plus)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1878 metadata_minus = self._normalize_metadata(metadata_minus)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1879 metadata = self._get_remote_metadata()

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1880 metadata.update(metadata_plus)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1881 for h in metadata_minus:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1882 if h in metadata:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1883 del metadata[h]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1884 src_bucket = self.bucket

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1885 # Boto prepends the meta prefix when adding headers, so strip prefix in

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1886 # metadata before sending back in to copy_key() call.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1887 rewritten_metadata = {}

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1888 for h in metadata:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1889 if (h.startswith('x-goog-meta-') or h.startswith('x-amz-meta-')):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1890 rewritten_h = (h.replace('x-goog-meta-', '')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1891 .replace('x-amz-meta-', ''))

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1892 else:

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1893 rewritten_h = h

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1894 rewritten_metadata[rewritten_h] = metadata[h]

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1895 metadata = rewritten_metadata

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1896 src_bucket.copy_key(self.name, self.bucket.name, self.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1897 metadata=metadata, preserve_acl=preserve_acl,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1898 headers=headers)

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1899

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1900 def restore(self, days, headers=None):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1901 """Restore an object from an archive.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1902

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1903 :type days: int

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1904 :param days: The lifetime of the restored object (must

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1905 be at least 1 day). If the object is already restored

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1906 then this parameter can be used to readjust the lifetime

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1907 of the restored object. In this case, the days

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1908 param is with respect to the initial time of the request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1909 If the object has not been restored, this param is with

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1910 respect to the completion time of the request.

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1911

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1912 """

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1913 response = self.bucket.connection.make_request(

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1914 'POST', self.bucket.name, self.name,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1915 data=self.RestoreBody % days,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1916 headers=headers, query_args='restore')

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1917 if response.status not in (200, 202):

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1918 provider = self.bucket.connection.provider

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1919 raise provider.storage_response_error(response.status,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1920 response.reason,

d67268158946 planemo upload commit a3f181f5f126803c654b3a66dd4e83a48f7e203b

bcclaywell

parents:

diff changeset

1921 response.read())

Mercurial > repos > bcclaywell > argo_navis

annotate venv/lib/python2.7/site-packages/boto/s3/key.py @ 0:d67268158946 draft