API Reference¶
This part of the documentation covers all the public classes in Depot.
Application Support¶
-
class
depot.manager.
DepotManager
¶ Takes care of managing the whole Depot environment for the application.
DepotManager tracks the created depots, the current default depot, and the WSGI middleware in charge of serving files for local depots.
While this is used to create the default depot used by the application it can also create additional depots using the
new()
method.In case you need to migrate your application to a different storage while keeping compatibility with previously stored file simply change the default depot through
set_default()
all previously stored file will continue to work on the old depot while new files will be uploaded to the new default one.-
classmethod
configure
(name, config, prefix='depot.')¶ Configures an application depot.
This configures the application wide depot from a settings dictionary. The settings dictionary is usually loaded from an application configuration file where all the depot options are specified with a given
prefix
.The default
prefix
is depot., the minimum required setting isdepot.backend
which specified the required backend for files storage. Additional options depend on the choosen backend.
-
classmethod
from_config
(config, prefix='depot.')¶ Creates a new depot from a settings dictionary.
Behaves like the
configure()
method but instead of configuring the application depot it creates a new one each time.
-
classmethod
get
(name=None)¶ Gets the application wide depot instance.
Might return
None
ifconfigure()
has not been called yet.
-
classmethod
get_default
()¶ Retrieves the current application default depot
-
classmethod
get_file
(path)¶ Retrieves a file by storage name and fileid in the form of a path
Path is expected to be
storage_name/fileid
.
-
classmethod
make_middleware
(app, **options)¶ Creates the application WSGI middleware in charge of serving local files.
A Depot middleware is required if your application wants to serve files from storages that don’t directly provide and HTTP interface like
depot.io.local.LocalFileStorage
anddepot.io.gridfs.GridFSStorage
-
classmethod
set_default
(name)¶ Replaces the current application default depot
-
classmethod
url_for
(path)¶ Given path of a file uploaded on depot returns the url that serves it
Path is expected to be
storage_name/fileid
.
-
classmethod
-
class
depot.middleware.
DepotMiddleware
(app, mountpoint='/depot', cache_max_age=604800, replace_wsgi_filewrapper=False)¶ WSGI Middleware in charge of serving Depot files.
Usually created using
depot.manager.DepotManager.make_middleware()
, it’s a WSGI middleware that serves files stored inside depots that do not provide a public HTTP url. For depot that provide a public url the request is redirected to the public url.In case you have issues serving files with your WSGI server your can try to set
replace_wsgi_filewrapper=True
which forces DEPOT to use its own internal FileWrapper instead of the one provided by your WSGI server.
Database Support¶
-
class
depot.fields.sqlalchemy.
UploadedFileField
(filters=(), upload_type=<class 'depot.fields.upload.UploadedFile'>, upload_storage=None, *args, **kw)¶ Provides support for storing attachments to SQLAlchemy models.
UploadedFileField
can be used as a Column type to store files into the model. The actual file itself will be uploaded to the default Storage, and only thedepot.fields.upload.UploadedFile
information will be stored on the database.The
UploadedFileField
is transaction aware, so it will delete every uploaded file whenever the transaction is rolled back and will delete any old file whenever the transaction is committed. This is the reason you should never associate the samedepot.fields.upload.UploadedFile
to two differentUploadedFileField
, otherwise you might delete a file already used by another model. It is usually best to just set thefile
orbytes
as content of the column and let theUploadedFileField
create thedepot.fields.upload.UploadedFile
by itself whenever it’s content is set.-
impl
¶ alias of
sqlalchemy.sql.sqltypes.Unicode
-
load_dialect_impl
(dialect)¶ Return a
TypeEngine
object corresponding to a dialect.This is an end-user override hook that can be used to provide differing types depending on the given dialect. It is used by the
TypeDecorator
implementation oftype_engine()
to help determine what type should ultimately be returned for a givenTypeDecorator
.By default returns
self.impl
.
-
process_bind_param
(value, dialect)¶ Receive a bound parameter value to be converted.
Custom subclasses of
_types.TypeDecorator
should override this method to provide custom behaviors for incoming data values. This method is called at statement execution time and is passed the literal Python data value which is to be associated with a bound parameter in the statement.The operation could be anything desired to perform custom behavior, such as transforming or serializing data. This could also be used as a hook for validating logic.
Parameters: - value – Data to operate upon, of any type expected by
this method in the subclass. Can be
None
. - dialect – the
Dialect
in use.
See also
types_typedecorator
_types.TypeDecorator.process_result_value()
- value – Data to operate upon, of any type expected by
this method in the subclass. Can be
-
process_result_value
(value, dialect)¶ Receive a result-row column value to be converted.
Custom subclasses of
_types.TypeDecorator
should override this method to provide custom behaviors for data values being received in result rows coming from the database. This method is called at result fetching time and is passed the literal Python data value that’s extracted from a database result row.The operation could be anything desired to perform custom behavior, such as transforming or deserializing data.
Parameters: - value – Data to operate upon, of any type expected by
this method in the subclass. Can be
None
. - dialect – the
Dialect
in use.
See also
types_typedecorator
_types.TypeDecorator.process_bind_param()
- value – Data to operate upon, of any type expected by
this method in the subclass. Can be
-
-
class
depot.fields.ming.
UploadedFileProperty
(filters=(), upload_type=<class 'depot.fields.upload.UploadedFile'>, upload_storage=None)¶ Provides support for storing attachments to Ming MongoDB models.
UploadedFileProperty
can be used as a field type to store files into the model. The actual file itself will be uploaded to the default Storage, and only thedepot.fields.upload.UploadedFile
information will be stored on the database.The
UploadedFileProperty
is UnitOfWork aware, so it will delete every uploaded file whenever unit of work is flushed and deletes a Document that stored files or changes the field of a document storing files. This is the reason you should never associate the samedepot.fields.upload.UploadedFile
to two differentUploadedFileProperty
, otherwise you might delete a file already used by another document. It is usually best to just set thefile
orbytes
as content of the column and let theUploadedFileProperty
create thedepot.fields.upload.UploadedFile
by itself whenever it’s content is set.Warning
As the Ming UnitOfWork does not notify any event in case it gets cleared instead of being flushed all the files uploaded before clearing the unit of work will be already uploaded but won’t have a document referencing them anymore, so DEPOT will be unable to delete them for you.
-
class
depot.fields.ming.
DepotExtension
(session)¶ Extends the Ming Session to track files.
Deletes old files when an entry gets removed or replaced, apply this as a Ming
SessionExtension
according to Ming documentation.-
after_flush
(obj=None)¶ After the session is flushed for
obj
If
obj
isNone
it means all the objects in the UnitOfWork which can be retrieved by iterating overODMSession.uow
-
before_flush
(obj=None)¶ Before the session is flushed for
obj
If
obj
isNone
it means all the objects in the UnitOfWork which can be retrieved by iterating overODMSession.uow
-
-
class
depot.fields.interfaces.
DepotFileInfo
(content, depot_name=None)¶ Keeps information on a content related to a specific depot.
By itself the DepotFileInfo does nothing, it is required to implement a
process_content()
method that actually saves inside the file info the information related to the content. The only information which is saved by default is the depot name itself.It is a specialized dictionary that provides also attribute style access, the dictionary parent permits easy encoding/decoding to most marshalling systems.
-
process_content
(content, filename=None, content_type=None)¶ Process content in the given depot.
This is implemented by subclasses to provide some kind of behaviour on the content in the related Depot. The default implementation is provided by
depot.fields.upload.UploadedFile
which stores the content into the depot.
-
-
class
depot.fields.interfaces.
FileFilter
¶ Interface that must be implemented by file filters.
File filters get executed whenever a file is stored on the database using one of the supported fields. Can be used to add additional data to the stored file or change it. When file filters are run the file has already been stored.
-
on_save
(uploaded_file)¶ Filters are required to provide their own implementation
-
-
class
depot.fields.upload.
UploadedFile
(content, depot_name=None)¶ Simple
depot.fields.interfaces.DepotFileInfo
implementation that stores files.Takes a file as content and uploads it to the depot while saving around most file information. Pay attention that if the file gets replaced through depot manually the
UploadedFile
will continue to have the old data.Also provides support for encoding/decoding using JSON for storage inside databases as a plain string.
- Default attributes provided for all
UploadedFile
include: - filename - This is the name of the uploaded file
- file_id - This is the ID of the uploaded file
- path - This is a depot_name/file_id path which can
- be used with
DepotManager.get_file()
to retrieve the file
- content_type - This is the content type of the uploaded file
- uploaded_at - This is the upload date in YYYY-MM-DD HH:MM:SS format
- url - Public url of the uploaded file
- file - The
depot.io.interfaces.StoredFile
instance of the stored file
-
process_content
(content, filename=None, content_type=None)¶ Standard implementation of
DepotFileInfo.process_content()
This is the standard depot implementation of files upload, it will store the file on the default depot and will provide the standard attributes.
Subclasses will need to call this method to ensure the standard set of attributes is provided.
- Default attributes provided for all
Filters¶
-
class
depot.fields.filters.thumbnails.
WithThumbnailFilter
(size=(128, 128), format='PNG')¶ Uploads a thumbnail together with the file.
Takes for granted that the file is an image. The resulting uploaded file will provide three additional properties named:
thumb_X_id
-> The depot file idthumb_X_path
-> Where the file is available in depotthumb_X_url
-> Where the file is served.
Where
X
is the resolution specified assize
in the filter initialization. By default this is(128, 128)
?so you will getthumb_128x128_id
,thumb_128x128_url
and so on.Warning
Requires Pillow library
Specialized FileTypes¶
-
class
depot.fields.specialized.image.
UploadedImageWithThumb
(content, depot_name=None)¶ Uploads an Image with thumbnail.
The default thumbnail format and size are PNG@128x128, those can be changed by inheriting the
UploadedImageWithThumb
and replacing thethumbnail_format
andthumbnail_size
class properties.The Thumbnail file is accessible as
.thumb_file
while the thumbnail url is.thumb_url
.Warning
Requires Pillow library
Storing Files¶
-
class
depot.io.interfaces.
StoredFile
(file_id, filename=None, content_type=None, last_modified=None, content_length=None)¶ Interface for already saved files.
- It provides metadata on the stored file through following properties:
- file_id
- filename
- content_type
- last_modified
- content_length
Already stored files can only be read back, so they are required to only provide
read(self, n=-1)
,close()
methods andclosed
property so that they can be read.To replace/overwrite a file content do not try to call the
write
method, instead use the storage backend to replace the file content.-
close
(*args, **kwargs)¶ Closes the file.
After closing the file it won’t be possible to read from it anymore. Some implementation might not do anything when closing the file, but they still are required to prevent further reads from a closed file.
-
closed
¶ Returns if the file has been closed.
When
closed
returnTrue
it won’t be possible to read anoymore from this file.
-
fileno
()¶ Returns underlying file descriptor if one exists.
An IOError is raised if the IO object does not use a file descriptor.
-
flush
()¶ Flush write buffers, if applicable.
This is not implemented for read-only and non-blocking streams.
-
isatty
()¶ Return whether this is an ‘interactive’ stream.
Return False if it can’t be determined.
-
name
¶ This is the filename of the saved file
If a filename was not available when the file was created this will return “unnamed” as filename.
-
next
¶
-
public_url
¶ The public HTTP url from which file can be accessed.
When supported by the storage this will provide the public url to which the file content can be accessed. In case this returns
None
it means that the file can only be served by theDepotMiddleware
itself.
-
read
(n=-1)¶ Reads
n
bytes from the file.If
n
is not specified or is-1
the whole file content is read in memory and returned
-
readable
()¶ Returns if the stored file is readable or not
Usually all stored files are readable
-
readline
()¶ Read and return a line from the stream.
If limit is specified, at most limit bytes will be read.
The line terminator is always b’n’ for binary files; for text files, the newlines argument to open can be used to select the line terminator(s) recognized.
-
readlines
()¶ Return a list of lines from the stream.
hint can be specified to control the number of lines read: no more lines will be read if the total size (in bytes/characters) of all lines so far exceeds hint.
-
seek
()¶ Change stream position.
Change the stream position to the given byte offset. The offset is interpreted relative to the position indicated by whence. Values for whence are:
- 0 – start of stream (the default); offset should be zero or positive
- 1 – current stream position; offset may be negative
- 2 – end of stream; offset is usually negative
Return the new absolute position.
-
seekable
()¶ Returns if the stored file is seekable or not
By default stored files are not seekable
-
tell
()¶ Return current stream position.
-
truncate
()¶ Truncate file to size bytes.
File pointer is left unchanged. Size defaults to the current IO position as reported by tell(). Returns the new size.
-
writable
()¶ Returns if the stored file is writable or not
Stored files are not writable, you should rely on the relative
FileStorage
to overwrite their content
-
class
depot.io.interfaces.
FileStorage
¶ Interface for storage providers.
The FileStorage base class declares a standard interface for storing and retrieving files in an underlying storage system.
Each storage system implementation is required to provide this interface to correctly work with filedepot.
-
create
(content, filename=None, content_type=None)¶ Saves a new file and returns the ID of the newly created file.
content
parameter can either bebytes
, anotherfile object
or acgi.FieldStorage
. Whenfilename
andcontent_type
parameters are not provided they are deducted from the content itself.
-
delete
(file_or_id)¶ Deletes a file. If the file didn’t exist it will just do nothing.
-
exists
(file_or_id)¶ Returns if a file or its ID still exist.
-
static
fileid
(file_or_id)¶ Gets the ID of a given
StoredFile
If the given parameter is already a StoredFile id it will directly return it.
-
static
fileinfo
(fileobj, filename=None, content_type=None, existing=None)¶ Tries to extract from the given input the actual file object, filename and content_type
This is used by the create and replace methods to correctly deduce their parameters from the available information when possible.
-
get
(file_or_id)¶ Opens the file given by its unique id.
This operation is guaranteed to return a
StoredFile
instance or should raiseIOError
if the file is not found.
-
list
()¶ Returns a list of file IDs that exist in the Storage.
Depending on the implementation there is the possibility that this returns more IDs than there have been created. Therefore this method is NOT guaranteed to be RELIABLE.
-
replace
(file_or_id, content, filename=None, content_type=None)¶ Replaces an existing file, an
IOError
is raised if the file didn’t already exist.Given a
StoredFile
or its ID it will replace the current content with the providedcontent
value. Iffilename
andcontent_type
are provided or can be deducted by thecontent
itself they will also replace the previous values, otherwise the current values are kept.
-
-
class
depot.io.local.
LocalFileStorage
(storage_path)¶ depot.io.interfaces.FileStorage
implementation that stores files locally.All the files are stored inside a directory specified by the
storage_path
parameter.-
create
(content, filename=None, content_type=None)¶ Saves a new file and returns the ID of the newly created file.
content
parameter can either bebytes
, anotherfile object
or acgi.FieldStorage
. Whenfilename
andcontent_type
parameters are not provided they are deducted from the content itself.
-
delete
(file_or_id)¶ Deletes a file. If the file didn’t exist it will just do nothing.
-
exists
(file_or_id)¶ Returns if a file or its ID still exist.
-
get
(file_or_id)¶ Opens the file given by its unique id.
This operation is guaranteed to return a
StoredFile
instance or should raiseIOError
if the file is not found.
-
list
()¶ Returns a list of file IDs that exist in the Storage.
Depending on the implementation there is the possibility that this returns more IDs than there have been created. Therefore this method is NOT guaranteed to be RELIABLE.
-
replace
(file_or_id, content, filename=None, content_type=None)¶ Replaces an existing file, an
IOError
is raised if the file didn’t already exist.Given a
StoredFile
or its ID it will replace the current content with the providedcontent
value. Iffilename
andcontent_type
are provided or can be deducted by thecontent
itself they will also replace the previous values, otherwise the current values are kept.
-
-
class
depot.io.gridfs.
GridFSStorage
(mongouri, collection='filedepot')¶ depot.io.interfaces.FileStorage
implementation that stores files on MongoDB.All the files are stored using GridFS to the database pointed by the
mongouri
parameter into the collection namedcollection
.-
create
(content, filename=None, content_type=None)¶ Saves a new file and returns the ID of the newly created file.
content
parameter can either bebytes
, anotherfile object
or acgi.FieldStorage
. Whenfilename
andcontent_type
parameters are not provided they are deducted from the content itself.
-
delete
(file_or_id)¶ Deletes a file. If the file didn’t exist it will just do nothing.
-
exists
(file_or_id)¶ Returns if a file or its ID still exist.
-
get
(file_or_id)¶ Opens the file given by its unique id.
This operation is guaranteed to return a
StoredFile
instance or should raiseIOError
if the file is not found.
-
list
()¶ Returns a list of file IDs that exist in the Storage.
Depending on the implementation there is the possibility that this returns more IDs than there have been created. Therefore this method is NOT guaranteed to be RELIABLE.
-
replace
(file_or_id, content, filename=None, content_type=None)¶ Replaces an existing file, an
IOError
is raised if the file didn’t already exist.Given a
StoredFile
or its ID it will replace the current content with the providedcontent
value. Iffilename
andcontent_type
are provided or can be deducted by thecontent
itself they will also replace the previous values, otherwise the current values are kept.
-
-
class
depot.io.boto3.
S3Storage
(access_key_id, secret_access_key, bucket=None, region_name=None, policy=None, storage_class=None, endpoint_url=None, prefix='')¶ depot.io.interfaces.FileStorage
implementation that stores files on S3.This is a version implemented on top of boto3. Installing
boto3
as a dependency is required to use this.All the files are stored inside a bucket named
bucket
onhost
which Depot connects to usingaccess_key_id
andsecret_access_key
.- Additional options include:
region
which can be used to specify the AWS region.endpoint_url
which can be used to specify an host different from Amazon AWS S3 Storagepolicy
which can be used to specify a canned ACL policy of eitherprivate
orpublic-read
.storage_class
which can be used to specify a class of storage.prefix
parameter can be used to store all files under specified prefix. Use a prefix like dirname/ (see trailing slash) to store in a subdirectory.
-
create
(content, filename=None, content_type=None)¶ Saves a new file and returns the ID of the newly created file.
content
parameter can either bebytes
, anotherfile object
or acgi.FieldStorage
. Whenfilename
andcontent_type
parameters are not provided they are deducted from the content itself.
-
delete
(file_or_id)¶ Deletes a file. If the file didn’t exist it will just do nothing.
-
exists
(file_or_id)¶ Returns if a file or its ID still exist.
-
get
(file_or_id)¶ Opens the file given by its unique id.
This operation is guaranteed to return a
StoredFile
instance or should raiseIOError
if the file is not found.
-
list
()¶ Returns a list of file IDs that exist in the Storage.
Depending on the implementation there is the possibility that this returns more IDs than there have been created. Therefore this method is NOT guaranteed to be RELIABLE.
-
replace
(file_or_id, content, filename=None, content_type=None)¶ Replaces an existing file, an
IOError
is raised if the file didn’t already exist.Given a
StoredFile
or its ID it will replace the current content with the providedcontent
value. Iffilename
andcontent_type
are provided or can be deducted by thecontent
itself they will also replace the previous values, otherwise the current values are kept.
-
class
depot.io.awss3.
S3Storage
(access_key_id, secret_access_key, bucket=None, host=None, policy=None, encrypt_key=False, prefix='')¶ depot.io.interfaces.FileStorage
implementation that stores files on S3.This is a version implemented on boto Installing
boto
as a dependency is required to use this.All the files are stored inside a bucket named
bucket
onhost
which Depot connects to usingaccess_key_id
andsecret_access_key
.- Additional options include:
host
which can be used to specify an host different from Amazon AWS S3 Storagepolicy
which can be used to specify a canned ACL policy of eitherprivate
orpublic-read
.encrypt_key
which can be specified to use the server side encryption feature.prefix
parameter can be used to store all files under specified prefix. Use a prefix like dirname/ (see trailing slash) to store in a subdirectory.
-
create
(content, filename=None, content_type=None)¶ Saves a new file and returns the ID of the newly created file.
content
parameter can either bebytes
, anotherfile object
or acgi.FieldStorage
. Whenfilename
andcontent_type
parameters are not provided they are deducted from the content itself.
-
delete
(file_or_id)¶ Deletes a file. If the file didn’t exist it will just do nothing.
-
exists
(file_or_id)¶ Returns if a file or its ID still exist.
-
get
(file_or_id)¶ Opens the file given by its unique id.
This operation is guaranteed to return a
StoredFile
instance or should raiseIOError
if the file is not found.
-
list
()¶ Returns a list of file IDs that exist in the Storage.
Depending on the implementation there is the possibility that this returns more IDs than there have been created. Therefore this method is NOT guaranteed to be RELIABLE.
-
replace
(file_or_id, content, filename=None, content_type=None)¶ Replaces an existing file, an
IOError
is raised if the file didn’t already exist.Given a
StoredFile
or its ID it will replace the current content with the providedcontent
value. Iffilename
andcontent_type
are provided or can be deducted by thecontent
itself they will also replace the previous values, otherwise the current values are kept.
-
class
depot.io.memory.
MemoryFileStorage
(**kwargs)¶ depot.io.interfaces.FileStorage
implementation that keeps files in memory.This is generally useful for caches and tests.
-
create
(content, filename=None, content_type=None)¶ Saves a new file and returns the ID of the newly created file.
content
parameter can either bebytes
, anotherfile object
or acgi.FieldStorage
. Whenfilename
andcontent_type
parameters are not provided they are deducted from the content itself.
-
delete
(file_or_id)¶ Deletes a file. If the file didn’t exist it will just do nothing.
-
exists
(file_or_id)¶ Returns if a file or its ID still exist.
-
get
(file_or_id)¶ Opens the file given by its unique id.
This operation is guaranteed to return a
StoredFile
instance or should raiseIOError
if the file is not found.
-
list
()¶ Returns a list of file IDs that exist in the Storage.
Depending on the implementation there is the possibility that this returns more IDs than there have been created. Therefore this method is NOT guaranteed to be RELIABLE.
-
replace
(file_or_id, content, filename=None, content_type=None)¶ Replaces an existing file, an
IOError
is raised if the file didn’t already exist.Given a
StoredFile
or its ID it will replace the current content with the providedcontent
value. Iffilename
andcontent_type
are provided or can be deducted by thecontent
itself they will also replace the previous values, otherwise the current values are kept.
-
Utilities¶
-
depot.io.utils.
file_from_content
(content)¶ Provides a real file object from file content
Converts
FileStorage
,FileIntent
andbytes
to an actual file.
-
class
depot.io.utils.
FileIntent
(fileobj, filename, content_type)¶ Represents the intention to upload a file
Whenever a file can be stored by depot, a FileIntent can be passed instead of the file itself. This permits to easily upload objects that are not files or to add missing information to the uploaded files.