Depot with Database¶
Depot provides built-in support for attachments to models, uploading a file and attaching it to a database entry is as simple as assigning the file itself to a model field.
Attaching to Models¶
Attaching files to models is as simple as declaring a field on the model itself,
support is currently provided for SQLAlchemy through the
depot.fields.sqlalchemy.UploadedFileField
and for Ming (MongoDB) through
the depot.fields.ming.UploadedFileProperty
:
from depot.fields.sqlalchemy import UploadedFileField
class Document(Base):
__tablename__ = 'document'
uid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(16), unique=True)
content = Column(UploadedFileField)
To actually store the file into the Document, assigning it to the content
property
is usually enough, just like files uploaded using FileStorage.create()
both
file objects, cgi.FieldStorage
and bytes
can be used:
# Store documents with attached files, the source can be a file or bytes
doc = Document(name=u'Foo',
content=open('/tmp/document.xls'))
DBSession.add(doc)
Note
In case of Python3 make sure the file is open in byte mode.
Depot will upload files to the default storage, to change where files are uploaded
use DepotManager.set_default()
.
Uploaded Files Information¶
Whenever a supported object is assigned to a UploadedFileField
or
UploadedFileProperty
it will be converted to a UploadedFile
object.
This is the same object you will get back when reloading the models from database and
apart from the file itself which is accessible through the .file
property, it provides
additional attributes described into the UploadedFile
documentation itself.
Most important property is probably the .url
property which provides an URL where the
file can be accessed in case the storage supports HTTP or the DepotMiddleware
is
available in your WSGI application.
Uploading on a Specific Storage¶
By default all the files are uploaded on the default
storage (the one
returned by DepotManager.get_default()
. This can be changed by
passing a upload_storage
argument explicitly to the database field declaration:
from depot.fields.sqlalchemy import UploadedFileField
class Document(Base):
__tablename__ = 'document'
uid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(16), unique=True)
content = Column(UploadedFileField(upload_storage='another_storage'))
If the specified upload_storage
is an alias to another storage, the
file will actually keep track of the real storage, so that when the alias
is switched to another storage, previously uploaded files continue to get
served from the old storage.
Session Awareness¶
Whenever an object is deleted or a rollback is performed the files uploaded during the unit of work or attached to the deleted objects are automatically deleted.
This is performed out of the box for SQLAlchemy, but requires the DepotExtension
to be registered as a session extension for Ming.
Note
Ming doesn’t currently provide an entry point for session clear, so files uploaded without a session flush won’t be deleted when the session is removed.
Custom Behaviour in Attachments¶
Often attaching a file to the model is not enough, if a video is uploaded you probably want to convert it to a supported format. Or if a big image is uploaded you might want to scale it down.
Most simple changes can be achieved using Filters, filters can create thumbnails of
an image or trigger actions when the file gets uploaded, multiple filters can be specified
as a list inside the filters
parameter of the column. More complex actions like
editing the content before it gets uploaded can be achieved subclassing
UploadedFile
and passing it as column upload_type
.
Attachment Filters¶
File filters are created by subclassing FileFilter
class, the only required
method to implement is FileFilter.on_save()
which you are required implement with
the actions you want to perform. The method will receive the uploaded file (after it already
got uploaded) and can add properties to it.
Inside filters the original content is available as a property of the uploaded file, by
accessing original_content
you can read the original content but not modify it, as
the file already got uploaded changing the original content has no effect.
If you need to store additional files, only use the UploadedFile.store_content()
method so that they are correctly tracked by the unit of work and deleted when the
associated document is deleted.
A filter that creates a thumbnail for an image would look like:
from depot.io import utils
from PIL import Image
from io import BytesIO
class WithThumbnailFilter(FileFilter):
def __init__(self, size=(128,128), format='PNG'):
self.thumbnail_size = size
self.thumbnail_format = format
def on_save(self, uploaded_file):
content = utils.file_from_content(uploaded_file.original_content)
thumbnail = Image.open(content)
thumbnail.thumbnail(self.thumbnail_size, Image.BILINEAR)
thumbnail = thumbnail.convert('RGBA')
thumbnail.format = self.thumbnail_format
output = BytesIO()
thumbnail.save(output, self.thumbnail_format)
output.seek(0)
thumb_file_name = 'thumb.%s' % self.thumbnail_format.lower()
# If you upload additional files do it with store_content
# to ensure they are correctly tracked by unit of work and
# removed on model deletion.
thumb_path, thumb_id = uploaded_file.store_content(output,
thumb_file_name)
thumb_url = DepotManager.get_middleware().url_for(thumb_path)
uploaded_file['thumb_id'] = thumb_id
uploaded_file['thumb_path'] = thumb_path
uploaded_file['thumb_url'] = thumb_url
To use it, just provide the filters
parameter in your UploadedFileField
or UploadedFileProperty
:
class Document(DeclarativeBase):
__tablename__ = 'docu'
uid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(16), unique=True)
photo = Column(UploadedFileField(filters=[WithThumbnailFilter()]))
As UploadedFile
remembers every value/attribute stored before saving it on
the database, all the thumb_id, thumb_path and thumb_url values will be available
when loading back the document:
>>> d = DBSession.query(Document).filter_by(name='Foo').first()
>>> print d.photo.thumb_url
/depot/default/5b1a489e-0d33-11e4-8e2a-0800277ee230
Custom Attachments¶
Filters are convenient and can be mixed together to enable multiple behaviours when a file is uploaded, but they have a limit: They cannot modify the uploaded file or the features provided when the file is retrieved from the database.
To avoid this limit users can specify their own upload type by subclassing
UploadedFile
. By specializing the UploadedFile.process_content()
method
it is possible to change the content before it’s stored and provide additional attributes.
Whenever the stored document is retrieved from the database, the file will be recovered
with the same type specified as the upload_type
, so any property or method provided
by the specialized type will be available also when the file is loaded back.
A possible use case for custom attachments is ensure an image is uploaded at a maximum resolution:
from depot.io import utils
from depot.fields.upload import UploadedFile
from depot.io.interfaces import FileStorage
from PIL import Image
from depot.io.utils import INMEMORY_FILESIZE
from tempfile import SpooledTemporaryFile
class UploadedImageWithMaxSize(UploadedFile):
max_size = 1024
def process_content(self, content, filename=None, content_type=None):
# As we are replacing the main file, we need to explicitly pass
# the filanem and content_type, so get them from the old content.
__, filename, content_type = FileStorage.fileinfo(content)
# Get a file object even if content was bytes
content = utils.file_from_content(content)
uploaded_image = Image.open(content)
if max(uploaded_image.size) >= self.max_size:
uploaded_image.thumbnail((self.max_size, self.max_size),
Image.BILINEAR)
content = SpooledTemporaryFile(INMEMORY_FILESIZE)
uploaded_image.save(content, uploaded_image.format)
content.seek(0)
super(UploadedImageWithMaxSize, self).process_content(content,
filename,
content_type)
Using it to ensure every uploaded image has a maximum resolution of 1024x1024 is as simple as passing it to the column:
class Document(DeclarativeBase):
__tablename__ = 'docu'
uid = Column(Integer, autoincrement=True, primary_key=True)
name = Column(Unicode(16), unique=True)
photo = Column(UploadedFileField(upload_type=UploadedImageWithMaxSize))
When saved the image will be automatically resized to 1024 when bigger than the maximum allowed size.