Mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Refactored scheduler
Date Tue, 04 Dec 2012 18:20:05 GMT
Hey guys,

I refactored the scheduler. What I did was the following:
1. Add a Scheduler trait and make everything depend on that, KafkaScheduler
implements this
2. Fix broken stuff in KafkaScheduler
3. Add support for one-time tasks (i.e. do this once in the future after a
specified delay.
4. Added a MockScheduler that also implements Scheduler

The mock scheduler is what I wanted to mention specifically. There really
should be no reason left to have timing dependencies or non-determinism in
tests now provided your code properly uses the Time and Scheduler
interfaces. Those who follow these things know this has been a pain point
for us.

Low Level Usage:
Here is how you use the MockScheduler for testing. MockScheduler has no
background threads and will not execute any of its tasks until you manually
call scheduler.tick(). tick() will check the Time instance it was
instantiated with and synchronously execute any tasks that need executing.

Normal Usage:
Most people don't even need to use the lower level tick() interface,
though. For convenience I added a scheduler instance in MockTime. This
scheduler will automatically tick every time that MockTime object advances
its clock. Here is an example that uses this:
  val counter = new AtomicInteger(0)
  val time = new MockTime()
  time.scheduler.schedule("my-counter-task", counter.getAndIncrement,
delay=30)
  assertEquals(0, counter.get)
  time.sleep(30)
  assertEquals(1, counter.get)

So when instantiating other things you are testing just pass in the
MockTime instance and the associated Scheduler instances and everything
should work like a charm.

I converted LogManagerTest to use this style of testing for its background
tasks if you want to see a real example.

Also, this is a pretty light patch intended for 0.8.1, so review it if you
get a chance: https://issues.apache.org/jira/browse/KAFKA-597

Cheers,

-Jay

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message