Bake-Off Results

The submissions were evaluated on three categories: throughput, traffic efficiency, and lines of code. The final rankings (at the bottom) are calculated from scores assigned to each sub-ranking. The “Throughput” category awards 30, 20, and 10 points for first, second, and third place. The “Traffic Efficiency” category awards 15, 10, and 5 points for first, second, and third place. Any ties are broken by the rankings in the source-code category.

Category 1: THROUGHPUT

No Error, Homogeneous Rankings:

  1. Dr. Nielson (700k+)
  2. Team 1 (525k)
  3. Team 2 (450k)

High Error, Homogeneous Rankings:

  1. Team 1 (320k)
  2. Dr. Nielson (300k)
  3. <no score for team2, failed>

Homogeneous, Overall Average:

  1. Dr. Nielson (430k)
  2. Team 1 (410k)
  3. <no score for team 2, failed>

No Error, Heterogeneous Rankings:

  1. Dr. Nielson (620k)
  2. Team 1 (590k)
  3. Team 2 (520k)

High Error, Heterogeneous Rankings:

  1. Team 1 (230k)
  2. Dr Nielson (200k)
  3. <no score for team 2, failed>

Overall, Heterogeneous Rankings:

  1. (TIE) Team 1 (400k)
  2. (TIE) Dr. Nielson (400k)
  3. <no score for team 2, failed>

Category 2: TRAFFIC EFFICIENCY

(Measured by packets routed per test)

No Error, Homogeneous Rankings:

  1. Dr. Nielson (1200)
  2. Team 1 (1600)
  3. Team 2 (3800)

High Error, Homogeneous Rankings:

  1. Team 1 (2900)
  2. Dr. Nielson (3500)
  3. <No score for team 2, failed>

Homogeneous, Average Packets Sent:

  1. Team 1 (2300)
  2. Dr. Nielson (2800)
  3. <No score for team 2, failed>

No Error, Heterogeneous Rankings:

  1. Team 1 (2200)
  2. Dr. Nielson (2300)
  3. Team 2 (2800)

High Error, Heterogeneous Rankings:

  1. Team 1 (4500)
  2. Dr. Nielson (5000)
  3. <No score for team 2, failed>

Overall, Heterogeneous Rankings:

  1. Team 1 (2950)
  2. Dr. Nielson (3200)
  3. <No score for team 2, failed>

Category 3: SOURCE CODE SIZE

(Lines of code for all .py files under Playground src)

  1. Team 2 (15,224)
  2. Team 1 (15,287)
  3. Dr. Nielson (15,737)

FINAL RANKINGS

  1. Team 1  235 (20+30+20+20+30+30 + 10+15+15+15+15+15)
  2. Dr. Nielson 225 (30+20+30+30+20+30 + 15+10+10+10+10+10)
  3. Team 2 30 (10+10 + 5+5)

 


Graphs:

  1. Dr. Nielson: Blue
  2. Team 1: Red
  3. Team 2: Green

Note that Team 2 failed many tests at the upper error rates, so their graphs look much better than they should.

Homogeneous Throughput

throughput_homo

Homogeneous Efficiency

traffic_homo

Heterogeneous Throughput

throughput

Heterogeneous Efficiency

traffic

Leave a Reply

Your email address will not be published. Required fields are marked *