Author: masnun

Python

Python: Writing custom log handler and formatter

Post author By masnun
Post date November 4, 2015
6 Comments on Python: Writing custom log handler and formatter

Log Handlers dictate how the log entries are handled. For example FileHandler would allow us to pipe our logs to a file. HTTP Handler makes it possible to send the logs over HTTP to a remote server. We can write our own log handlers if we need to customize the way our logs are processed. Writing a custom handler is pretty simple. We have to subclass it from logging.Handler class and must define the emit method. This method is called with each log record so we can process it.

Here’s an exmaple of a custom log handler which POSTs the logs to a remote server using the popular requests library.

class RequestsHandler(Handler):
    def emit(self, record):
        log_entry = self.format(record)
        return requests.post('http://example.com:8080/',
                             log_entry, headers={"Content-type": "application/json"}).content

class RequestsHandler(Handler):

def emit(self, record):

log_entry = self.format(record)

return requests.post('http://example.com:8080/',

log_entry, headers={"Content-type": "application/json"}).content

Now we have our custom handler that will post the logs to another server. What if we want to send the message in a specific format? We write our custom formatter. A formatter has a format method which gets the record. We can take the record and return a message formatted according our need. Here’s an example for logstash format:

class LogstashFormatter(Formatter):
    def __init__(self, task_name=None):
        self.task_name = task_name

        super(LogstashFormatter, self).__init__()

    def format(self, record):
        data = {'@message': record.msg,
                '@timestamp': datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%fZ'),
                '@type': 'IronWorker'}

        if self.task_name:
            data['@task_name'] = self.task_name

        return json.dumps(data)

class LogstashFormatter(Formatter):

def __init__(self, task_name=None):

self.task_name = task_name

super(LogstashFormatter, self).__init__()

def format(self, record):

data = {'@message': record.msg,

'@timestamp': datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S.%fZ'),

'@type': 'IronWorker'}

if self.task_name:

data['@task_name'] = self.task_name

return json.dumps(data)

So we have got ourselves a custom log handler and a formatter. But how do we use them? Here’s a short code snippet:

    def __init_logging(self):
        self.logger = logging.getLogger(self.logger_name.upper())
        self.logger.setLevel(self.logging_level)

        handler = RequestsHandler()
        formatter = LogstashFormatter(self.logger_name.upper())
        handler.setFormatter(formatter)
        self.logger.addHandler(handler)

def __init_logging(self):

self.logger = logging.getLogger(self.logger_name.upper())

self.logger.setLevel(self.logging_level)

handler = RequestsHandler()

formatter = LogstashFormatter(self.logger_name.upper())

handler.setFormatter(formatter)

self.logger.addHandler(handler)

Django Python

Django REST Framework: Displaying full URL for ImageField or FileField

Post author By masnun
Post date October 26, 2015
10 Comments on Django REST Framework: Displaying full URL for ImageField or FileField

If you have any ImageField or FileField in your model, you can easily display the full URL (including the hostname/domain) for the file/image. Django REST Framework’s model serializers would do it for you. However, to get the hostname/domain name, the serializer needs a “request context” so it can infer the necessary parts and build a full url.

So if you’re manually invoking a serializer, please pass a request context like this:

serialized_store = StoreSerializer(store, context={"request": request})

1	serialized_store = StoreSerializer(store, context={"request": request})

If you use ModelViewSet, DRF will automatically pass the request context while initializing the serializer. So in that case you don’t need to do anything. You need to pass the context only when you’re manually creating a serializer instance.

Django Python

Django REST Framework: Dynamic Fields in Serializers

Post author By masnun
Post date October 21, 2015
7 Comments on Django REST Framework: Dynamic Fields in Serializers

Here’s the use case, I need to add some extra fields to my serializer depending on a request. For example, if the query string contains “extra=true”, then add the extra fields. Luckily, serializers get a “context” argument when they are initialized. We can use this to customize our serializers as needed.

The fields defined on a serializer are available in a instance variable named “fields”. So we can add/delete/edit fields from it. Let’s see an example:

class OrderSerializer(ModelSerializer):
    payments = PaymentSerializer(many=True, read_only=True)
    evaluations = OrderEvaluationSerializer(many=True, read_only=True)

    def __init__(self, *args, **kwargs):
        super(OrderSerializer, self).__init__(*args, **kwargs)

        request = kwargs['context']['request']
        include_address = request.GET.get('include_address', False)

        if include_address:
            self.fields['user_location'] = UserLocationSerializer(read_only=True)

class OrderSerializer(ModelSerializer):

payments = PaymentSerializer(many=True, read_only=True)

evaluations = OrderEvaluationSerializer(many=True, read_only=True)

def __init__(self, *args, **kwargs):

super(OrderSerializer, self).__init__(*args, **kwargs)

request = kwargs['context']['request']

include_address = request.GET.get('include_address', False)

if include_address:

self.fields['user_location'] = UserLocationSerializer(read_only=True)

Please note, here we have used another serializer (UserLocationSerializer) from inside our main serializer. The second one is being initialized by us. So it would not get the context. If we need to do something down there as well, then we need to pass it ourselves.

class OrderSerializer(ModelSerializer):
    payments = PaymentSerializer(many=True, read_only=True)
    evaluations = OrderEvaluationSerializer(many=True, read_only=True)

    def __init__(self, *args, **kwargs):
        super(OrderSerializer, self).__init__(*args, **kwargs)

        request = kwargs['context']['request']
        include_address = request.GET.get('include_address', False)

        if include_address:
            self.fields['user_location'] = UserLocationSerializer(read_only=True, context=kwargs['context'])

class OrderSerializer(ModelSerializer):

payments = PaymentSerializer(many=True, read_only=True)

evaluations = OrderEvaluationSerializer(many=True, read_only=True)

def __init__(self, *args, **kwargs):

super(OrderSerializer, self).__init__(*args, **kwargs)

request = kwargs['context']['request']

include_address = request.GET.get('include_address', False)

if include_address:

self.fields['user_location'] = UserLocationSerializer(read_only=True, context=kwargs['context'])

Now the second serializer will get the request too and we can use the same way to customize it!

Recent Posts

Recent Comments

Archives

Categories