Mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Arthur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-657) Add an API to commit offsets
Date Wed, 12 Dec 2012 18:16:19 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530174#comment-13530174
] 

David Arthur commented on KAFKA-657:
------------------------------------

Thanks [~jkreps], that clears things up quite a bit. Another question I have is around the
request envelope (clientId, correlationId, etc).

I understand correlationId is used to allow multiplexing requests/response, but what about
replicaId, clientId, etc. I mostly copied these from other Request classes - a bit of cargo-cult
programming I guess :)

{code}
    val versionId = buffer.getShort
    val correlationId = buffer.getInt
    val clientId = readShortString(buffer)
    val replicaId = buffer.getInt
{code}

Are all of these necessary for the OffsetCommitRequest/Response? Specifically, is replicaId
necessary?
                
> Add an API to commit offsets
> ----------------------------
>
>                 Key: KAFKA-657
>                 URL: https://issues.apache.org/jira/browse/KAFKA-657
>             Project: Kafka
>          Issue Type: New Feature
>            Reporter: Jay Kreps
>              Labels: project
>         Attachments: KAFKA-657v1.patch
>
>
> Currently the consumer directly writes their offsets to zookeeper. Two problems with
this: (1) This is a poor use of zookeeper, and we need to replace it with a more scalable
offset store, and (2) it makes it hard to carry over to clients in other languages. A first
step towards accomplishing that is to add a proper Kafka API for committing offsets. The initial
version of this would just write to zookeeper as we do today, but in the future we would then
have the option of changing this.
> This api likely needs to take a sequence of consumer-group/topic/partition/offset entries
and commit them all.
> It would be good to do a wiki design on how this would work and consensus on that first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message