Generic API resources¶
graceful provides you with some set of generic resources in order to help you describe how structured is data in your API. All of them expect that some serializer instance is provided as a class level attribute. Serializer will handle describing resource fields and also translation between resource representation and internal object values that you use inside of your application.
RetrieveAPI¶
RetrieveAPI
represents single element serialized resource. In ‘content’
section of GET response it will return single object. On OPTIONSrequest
it will return additional field named ‘fields’ that describes all serializer
fields.
It expects from you to implement .retrieve(self, params, meta, **kwargs)
method handler that retrieves single object (e.g. from some storage) that will
be later serialized using provided serializer.
retrieve()
accepts following arguments:
- params (dict): dictionary of parsed parameters accordingly to definitions provided as resource class atributes.
- meta (dict): dictionary of meta parameters anything added to this dict will will be later included in response ‘meta’ section. This can already prepopulated by method that calls this handler.
- kwargs (dict): dictionary of values retrieved from route url template by falcon. This is suggested way for providing resource identifiers.
Example usage:
db = SomeDBInterface()
api = application = falcon.API()
class FooResource(RetrieveAPI):
serializer = RawSerializer()
def retrieve(self, params, meta, foo_id, **kwargs):
return db.Foo.get(id=foo_id)
# note url template param that will be passed to `FooResource.get_object()`
api.add_route('foo/{foo_id}', FooResource())
RetrieveUpdateAPI¶
RetrieveUpdateAPI
extends RetrieveAPI
with capability to
update objects with new data from resource representation provided in
PUT request body.
It expects from you to implement same handlers as for RetrieveAPI
and also new .update(self, params, meta, validated, **kwargs)
method handler
that updates single object (e.g. in some storage). Updated object may or may
not be returned in response ‘content’ section (this is optional)
update()
accepts following arguments:
- params (dict): dictionary of parsed parameters accordingly to definitions provided as resource class atributes.
- meta (dict): dictionary of meta parameters anything added to this dict will will be later included in response ‘meta’ section. This can already prepopulated by method that calls this handler.
- validated (dict): dictionary of internal object fields values after converting from representation with full validation performed accordingly to definition contained within serializer instance.
- kwargs (dict): dictionary of values retrieved from route url template by falcon. This is suggested way for providing resource identifiers.
If update will return any value it should have same form as return value
of retrieve()
because it will be again translated into representation
with serializer.
Example usage:
db = SomeDBInterface()
api = application = falcon.API()
class FooResource(RetrieveUpdateAPI):
serializer = RawSerializer()
def retrieve(self, params, meta, foo_id, **kwargs):
return db.Foo.get(id=foo_id)
def update(self, params, meta, foo_id, **kwargs):
return db.Foo.update(id=foo_id)
# note: url template kwarg that will be passed to
# `FooResource.get_object()`
api.add_route('foo/{foo_id}', FooResource())
RetrieveUpdateDeleteAPI¶
RetrieveUpdateDeleteAPI
extends RetrieveUpdateAPI
with
capability to delete objects using DELETE requests.
It expects from you to implement same handlers as for RetrieveUpdateAPI
and also new .delete(self, params, meta, **kwargs)
method handler
that deletes single object (e.g. in some storage).
delete()
accepts following arguments:
- params (dict): dictionary of parsed parameters accordingly to definitions provided as resource class atributes.
- meta (dict): dictionary of meta parameters anything added to this dict will will be later included in response ‘meta’ section. This can already prepopulated by method that calls this handler.
- kwargs (dict): dictionary of values retrieved from route url template by falcon. This is suggested way for providing resource identifiers.
Example usage:
db = SomeDBInterface()
api = application = falcon.API()
class FooResource(RetrieveUpdateAPI):
serializer = RawSerializer()
def retrieve(self, params, meta, foo_id, **kwargs):
return db.Foo.get(id=foo_id)
def update(self, params, meta, foo_id, **kwargs):
return db.Foo.update(id=foo_id)
def delete(self, params, meta, **kwargs):
db.Foo.delete(id=foo_id)
# note url template param that will be passed to `FooResource.get_object()`
api.add_route('foo/{foo_id}', FooResource())
ListAPI¶
ListAPI
represents list of resource instances. In ‘content’
section of GET response it will return list of serialized objects
representations. On OPTIONS request it will return additional
field named ‘fields’ that describes all serializer fields.
It expects from you to implement .list(self, params, meta, **kwargs)
method handler that retrieves list (or any iterable) of objects
(e.g. from some storage) that will be later serialized using provided
serializer.
list()
accepts following arguments:
- params (dict): dictionary of parsed parameters accordingly to definitions provided as resource class atributes.
- meta (dict): dictionary of meta parameters anything added to this dict will will be later included in response ‘meta’ section. This can already prepopulated by method that calls this handler.
- kwargs (dict): dictionary of values retrieved from route url template by falcon. This is suggested way for providing resource identifiers.
Example usage:
db = SomeDBInterface()
api = application = falcon.API()
class FooListResource(ListAPI):
serializer = RawSerializer()
def list(self, params, meta, **kwargs):
return db.Foo.all(id=foo_id)
# note that in most cases there is no need do define
# variables in url template for list type of resources
api.add_route('foo/', FooListResource())
ListCreateAPI¶
ListCreateAPI
extends ListAPI
with capability to
create new objects with data from resource representation provided in
POST or PATCH request body.
It expects from you to implement same handlers as for ListAPI
and also new .create(self, params, meta, validated, **kwargs)
and (optionally) .create_bulk(self, params, meta, validated, **kwargs)
method handlers that are able to create single single and multiple objects
(e.g. in some storage). Created object may or may not be returned in response
‘content’ section (this is optional)
create()
accepts following arguments:
- params (dict): dictionary of parsed parameters accordingly to definitions provided as resource class atributes.
- meta (dict): dictionary of meta parameters anything added to this dict will will be later included in response ‘meta’ section. This can already prepopulated by method that calls this handler.
- validated (dict): a single dictionary of internal object fields values after converting from representation with full validation performed accordingly to definition contained within serializer instance.
- kwargs (dict): dictionary of values retrieved from route url template by falcon. This is suggested way for providing resource identifiers.
create_bulk()
accepts following arguments:
- params (dict): dictionary of parsed parameters accordingly to definitions provided as resource class atributes.
- meta (dict): dictionary of meta parameters anything added to this dict will will be later included in response ‘meta’ section. This can already prepopulated by method that calls this handler.
- validated (dict): a list of multiple dictionaries of internal objects’ field values after converting from representation with full validation performed accordingly to definition contained within serializer instance.
- kwargs (dict): dictionary of values retrieved from route url template by falcon. This is suggested way for providing resource identifiers.
If create()
and create_bulk()
return any value then it should have
same form compatible with the return value of retrieve()
because it will
be again translated into representation with serializer. Of course create()
should return single instance of resource but create_bulk()
should return
collection of resources.
Note that default implementation of ListCreateAPI.create_bulk()
is very
simple and may not be suited for every use case. If you want to use it please
refer to Guide for creating resources in bulk.
Example usage:
db = SomeDBInterface()
api = application = falcon.API()
class FooListResource(ListCreateAPI):
serializer = RawSerializer()
def list(self, params, meta, **kwargs):
return db.Foo.all(id=foo_id)
def create(self, params, meta, validated, **kwargs):
return db.Foo.create(**validated)
# note that in most cases there is no need do define
# variables in url template for list type of resources
api.add_route('foo/', FooListResource())
Paginated generic resources¶
PaginatedListAPI
and PaginatedListCreateAPI
are versions
of ListAPI
and ListAPI
classes that support simple pagination
with following parameters:
- page_size: size of a single response page
- page: page count
They also will ‘meta’ section with following information on GET requests:
page_size
page
next
- url query string for next page (only ifmeta['is_more']
exists and isTrue
)prev
- url query string for previous page (None
if first page)
Paginated variations of generic list resource do not assume anything about
your resources so actual pagination must still be implemented inside of
list()
handlers. Anyway this class allows you to manage params and meta
for pagination in consistent way across all of your resources if you only
decide to use it:
db = SomeDBInterface()
api = application = falcon.API()
class FooPaginatedResource(PaginatedListAPI):
serializer = RawSerializer()
def list(self, params, meta, **kwargs):
query = db.Foo.all(id=foo_id).offset(
params['page'] * params['page_size']
).limit(
params['page_size']
)
# use meta['has_more'] to find out if there are
# any pages behind this one
if db.Foo.count() > (params['page'] + 1) * params['page_size']:
meta['has_more'] = True
return query
api.add_route('foo/', FooPaginatedtResource())
Note
If you don’t like anything about this opinionated meta section that
paginated generic resources provide, you can always override it with
own add_pagination_meta(params, meta)
method handler.
Generic resources without serialization¶
If you don’t like how serializers work there are also two very basic generic
resources that does not rely on serializers: Resource
and
ListResource
. They can be extended with mixins found in
graceful.resources.mixins
module and provide the same method handlers
like the generic resources that utilize serializers (i.e. list()
,
retrieve()
, update()
and so on). Note that they do not perform anything
beyond content-type level serialization.
Guide for creating resources in bulk¶
ListCreateAPI
ships with default implementation of create_bulk()
method that will call the create()
method separately for every resource
instance retrieved from request payload. The actual code is following:
def create_bulk(self, params, meta, **kwargs):
validated = kwargs.pop('validated')
return [self.create(params, meta, validated=item) for item in validated]
This approach to bulk resource creation may not be the most performant one if
you save resource instance to your storage on every create()
call.
The other concern is whether you care about data consistency in your storage
and want to ensure the “all or nothing” semantics. With default bulk creation
handler it may be hard to enforce such contraints. Anyway, you can easily
override this method to suit your own needs.
There are at least three ways you can handle bulk resource creation in graceful:
- Completely separate bulk and single resource creation: allow
create()
andcreate_bulk()
handlers to have their own separate code responsible for saving data in the storage. - Deffered saves: Allow your
create()
handler to skip saves if specific keyword parameter is set and then do your saves in thcreate_bulk()
handler. - Utilize your storage transactions: Wrap your data processing with per-request transaction to ensure “all or nothing” semantics on database level.
Completely separate bulk and single resource creation¶
This approach is simplest to implement but makes only sense if the process of your resource creation is very simple and heavily relies on serializers to validate and prepare your data before save.
Assume your API allows to create and retrieve simple documents in some simple storage that may even not be a real database. Good example would be an API dealing with Solr search engine:
from pysolr import Solr
from graceful.serializers import BaseSerializer
from graceful.fields import StringField
from graceful.resources.generic import ListCreateAPI
solr = Solr("<solr url>", "<solr port>")
class DocumentSerializer(BaseSerializer):
text = StringField("Document content")
author = StringField(
"Document author",
# note: Assume that due to legacy reasons this field
# is stored under different name in Solr.
# graceful is great in dealing with such problems!
source="autor_name_t"
)
class DocumentsAPI(ListCreateAPI):
def list(self, params, meta, **kwargs):
return solr.search("*:*")
def create(self, params, meta, validated, **kwargs):
solr.add([validated])
# note: return document back so its representation
# can be included in response body
return validated
Solr search engine is especially good example here because it will not handle
well multiple single-ducument save requests and the best approach is to
batch them. The pysolr
module (popular library for integration with solr)
allows you to save multiple documents with single Solr.add()
call.
Actually, it even encourages you to batch documents using single call because
it accepts only list as input argument.
Let’s override the default create_bulk()
so it will save all the documents
it receives as the validated
argument without calling create()
handler:
class DocumentsAPI(ListCreateAPI):
def list(self, params, meta, **kwargs):
return solr.search("*:*")
def create(self, params, meta, validated, **kwargs):
solr.add([validated])
# note: return document back so its representation
# can be included in the response body
return validated
def create_bulk(self, params, meta, validated, **kwargs):
solr.add(validated)
# note: return documents back so their representation
# can be included in the response body
return validated
Note that above technique works best for simple use cases where the
validated
argument represents complete data that can be easily saved
directly to your storage without any further modification.
If you need any additional processing of resources in your custom create()
and create_bulk()
methods before saving them to your storage,
the code can quickly become hard to mantain. Anyway, you can start with this
approach and refactor it later into deferred saves pattern as these two are
very alike and offer similar advantages.
Deferred saves¶
In previous section we said that having separate code that independently saves single resource and resources in bulk may not be a best approach if you need to make some additional data processing before saves. No matter if you do a non-serializer-based data validation or talk to some other external services, you will need to duplicate this additional processing code in both handlers. With proper approach you can limit the code duplication by extrating your resource processing procedures to additial methods but it will eventually make things unnecessarily complex and will still be hard to maintain.
A little improvement to previous code is to reuse single resource creation
handler in your custom create_bulk()
implementation but allow the
create()
handler to skip saving data to storage on the caller’s demand.
Thus any per-resource processing will always stay in the create()
handler
code and the create_bulk()
will be responsible only for saving the data in
bulk:
class DocumentsAPI(ListCreateAPI):
def list(self, params, meta, **kwargs):
return solr.search("*:*")
def create(self, params, meta, validated, skip_save=False, **kwargs):
# do some additional processing like adding defaults etc.
validated['created_at'] = time.time()
# note: skip_save defaults to False on ordinary POST requests
# this means ``create()`` was called in single-resource mode
if not skip_save:
solr.add([validated])
# note: return document back so its representation
# can be included in the response body
return validated
def create_bulk(self, params, meta, validated, **kwargs):
validated = kwargs.pop('validated')
processed = [
self.create(params, meta, item, skip_save=True)
for item in validated
]
solr.add(processed)
return processed
This way you can be sure that anything you add to the create()
handler
will also affect the resources created in bulk. Additionally your API is more
efficient because it can save the data in bulk with single request to your
storage backend instead of making multiple requests.
Utilize your storage transactions¶
Sometimes you may not concerned about the performance of multiple small saves
but only want to have the “all or nothing” semantics of the bulk creation
method. If the integration with your storage backend allows you to enforce
transactions on the block of code you can easily use such feature to make sure
that all the separate saves done with create()
handler will take effect
in the “all or nothing” manner. Good use case for such appoach could be working
with any RDBMS that allows to use transactions.
Let’s assume you have a per-request session
object that wraps the
integration with the storage backend and allows you to set savepoints and
commit/rollback transactions. Many ORM layers (e.g. SQLAlchemy) offer such
kind of object code for such technique may look very simillar for different
storage providers:
# note: example sqlachemy integration could work that way
engine = create_engine("...")
Session = sessionmaker(bind=engine)
class MyAPI(ListCreateAPI):
def on_post(req, resp, **kwargs):
# inject session object into kwargs so it can be later
# used by ``create()`` handler to manipulate storage
# and manage transaction
session = Session()
try:
super().on_post(req, resp, session=session, **kwargs)
except:
session.rollback()
raise
else:
session.commit()
def on_patch(req, resp, **kwargs):
# inject session object into kwargs so it can be later
# used by ``create_bulk()`` handler to manipulate storage
# and manage transaction
session = Session()
try:
super().on_patch(req, resp, session=session, **kwargs)
except:
session.rollback()
raise
else:
session.commit()