This release mostly solves problems related to message deserialization and normalizes some of the naming conventions to ease during the upgrade to the upcoming 2.0
version.
Note: This release is the last release with ruby-kafka
under the hood. We’ve already started the process of moving to rdkafka-ruby
.
Note: If you are using Sidekiq-Backend
plugin, please make sure that you’ve processed all the jobs from your Sidekiq queue before upgrading Karafka gems.
consumer#metadata
is now consumer#batch_metadata
This change is trivial: if you use batch consuming mode and you use the Consumer#metadata
method, replace it with Consumer#batch_metadata
.
# Karafka 1.3 class UsersConsumerMessage
metadata available under#metadata
methodUp to version
1.3
, all the message metadata would be directly available under the root scope of theparams
object using both direct method reference as well as with#[]
accessor.While it felt like “The Rails way”, it had several side-effects, amongst which the biggest were the need of having a hash like API, issues with accessing metadata without payload deserialization, and a lack of clear separation between payload and the metadata.
From now on, you can use the
params.metadata
object to fetch all the metadata.Note: we’ve preserved the direct metadata values fetching from the
params
object to preserve backwards compatibility. # 1.3 params['partition'] #=> 0 params.partition #=> 0 # 1.4 params['partition'] #=> NoMethodError (undefined method '[]') # This will work due to backward compatibility params.partition #=> 0 # This is the recommended way of accessing metadata params.metadata.partition #=> 0 # This will also work as metadata is a struct now params.metadata[:partition] #=> 0 params.metadata['partition'] #=> 0Message metadata access allowed without message deserialization
When accessing metadata, the payload is not being deserialized until
#payload
method is being used.null message support in the default JSON deserializer
When the Kafka message payload is
null
/nil
, deserialization won’t fail. Support for it was added as some of the Karafka users use log compaction with anil
payload. In case like that,#payload
will returnnil
.
Karafka::Params::Params
no longer inherits from aHash
Karafka::Params::Params
is now just a struct. This change is introduced to normalize the setup, limit the corner cases and simplify the interface only to methods that are really needed.Documentation
Our Wiki has been updated accordingly to the
1.4
status. Please notify us if you find any incompatibilities.Getting started with Karafka
If you want to get started with Kafka and Karafka as fast as possible, then the best idea is to just clone our example repository:
git clone https://github.com/karafka/example-app ./example_appthen, just bundle install all the dependencies:
cd ./example_app bundle installand follow the instructions from the example app Wiki.
The post Karafka framework 1.4.0 Release Notes (Ruby + Kafka) appeared first on Running with Ruby.
Welcome to the Ubuntu Weekly Newsletter, Issue 883 for the week of March 9 –…
In this article, we will see how to install nvidia-smi on Ubuntu or Debian Linux.…
In this article, we will see how to install clang tool on Ubuntu or Debian…
When working with Docker containers on Raspberry Pi devices, you might encounter frustrating signature verification…
You’ve recently upgraded to Ubuntu 18.04 and found that your OpenVPN connection no longer resolves…
Have you ever tried to open System Monitor on your Ubuntu 18.04 system only to…