Mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: order guarantee failure: better/best effort?
Date Thu, 13 Dec 2012 16:42:21 GMT
So my belief is that we handle this correctly and that I was mistaken in
filing that bug. However, since, as the bug points out, the problem can't
exist in the scala client this may be mistaken and a problem may well
remain.

There are two reasons that requests should have enforced ordering:
1. The scala client blocks until it gets a response. As you say, this
doesn't help you.
2. The server only processes a single request per socket at a time.

Can you confirm that (1) these messages are sent to the same partition
(guarantee is only within a partition), (2) the requests are sent over the
same socket (order can only be guaranteed over a single connection). If
these are true and we are re-ordering I think we have a bug...if so if you
could give a small portable program of some kind that would reproduce the
problem that would be awesome since I don't think the scala client will.

-Jay


On Thu, Dec 13, 2012 at 12:01 AM, ben fleis <ben.fleis@gmail.com> wrote:

> Hello all,
>
> While running my own system tests against 0.8/HEAD, I was seeing repeated
> ordering failures, and tracked it down a bit further.  In simple summary, I
> am sending out 2 consecutive ProduceRequests, X and Y.  Y gets written to
> disk before X.  Consumer sees Y before X.
>
> Bug #382 <https://issues.apache.org/jira/browse/KAFKA-382> discusses this
> possibility, and it appears to describe my issue.  My client is, like any
> normal client written in Node, asynchronous by nature.  I can add waits to
> make it synchronous, but this feels like it fails the "best effort" test.
>  I.e., it seems reasonable that a pipelining client running without errors
> ought to maintain order, all the time.
>
> Thoughts?
>
> b
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message