Random write timeout for rrdtool plugin

Hi,

i was thinking how to "spread out" writes to rrd files a bit, because
now its big spike every CacheTimeout or little smaller "square" on
graph if u use WritesPerSecond. So ive written little patch which
"spreads out" writing by changing Cache timeout every time rrdtool
plugin finds data to save. Basically instead of moving data older than
CacheTimeout to write queue it moves it if its older than CacheTimeout
+- RandomTimeout. What it changes?

Without it, gathered data is "synchronised" with eachother, for
example (CacheTimeout = 600):
1.collectd starts
2. after 10 minutes, data from all plugins get "too old" and get
pushed into write queue and get saved
3. after another 10 minutes, same thing, all data "ages" at same time
and get saved in one big chunk

With it (RandomTimeout=300) it works like that
1. collectd starts
2. after 5 minutes some data (lets call it A) starts to go into write queue
3. after 10 minutes from start about 50% (on average) data is saved
(lets call it B)
4. finally, after 15 minutes, all "leftover" data gets saved (lets call it C)
5. next "cycle"
6. data A ages first (cos it was put to disk first) and like before,
some of it gets writen earlier, some of it gets written later)
7. after that data B ages and like before writes are spread over 10 mins
8. same with C

so first cycle (looking at i/o) looks like sinus, next 10 minute cycle
is same sinus but flattened a bit and so on (looks like fading sinus),
and after few cycles it gives pretty much same amount on writes per
sec, no ugly spikes.
Effect looks like that:
http://img24.imageshack.us/img24/7294/drrawcgi.png
(after few more h it will be more "smooth")

Regards
Mariusz

Signed-off-by: Florian Forster <octo@huhu.verplant.org>

diff --git a/src/rrdtool.c b/src/rrdtool.c
index c63f883afea9b86d8138472230b0f7fa3d1e8999..2787944330b28959ec81d8c25a9e9d90d9fe6f23 100644 (file)
--- a/src/rrdtool.c
+++ b/src/rrdtool.c
@@ -76,7 +76,8 @@ static const char *config_keys[] =
        "RRARows",
        "RRATimespan",
        "XFF",
-       "WritesPerSecond"
+       "WritesPerSecond",
+       "RandomTimeout"
 };
 static int config_keys_num = STATIC_ARRAY_SIZE (config_keys);
 
@@ -103,6 +104,8 @@ static rrdcreate_config_t rrdcreate_config =
  * ALWAYS lock `cache_lock' first! */
 static int         cache_timeout = 0;
 static int         cache_flush_timeout = 0;
+static int         random_timeout = 1;
+static int         random_timeout_mod = 1;
 static time_t      cache_flush_last;
 static c_avl_tree_t *cache = NULL;
 static pthread_mutex_t cache_lock = PTHREAD_MUTEX_INITIALIZER;
@@ -739,7 +742,8 @@ static int rrd_cache_insert (const char *filename,
                        filename, rc->values_num,
                        (unsigned long)(rc->last_value - rc->first_value));
 
-       if ((rc->last_value - rc->first_value) >= cache_timeout)
+
+       if ((rc->last_value - rc->first_value) >= (cache_timeout + (random_timeout - (rand() % random_timeout_mod) ) ) )
        {
                /* XXX: If you need to lock both, cache_lock and queue_lock, at
                 * the same time, ALWAYS lock `cache_lock' first! */
@@ -988,6 +992,19 @@ static int rrd_config (const char *key, const char *value)
                        write_rate = 1.0 / wps;
                }
        }
+       else if (strcasecmp ("RandomTimeout", key) == 0)
+        {
+               random_timeout = atoi (value);
+               if( random_timeout < 0 )
+               {
+               fprintf (stderr, "rrdtool: `RandomTimeout' must "
+                        "be greater than or equal to zero.\n");
+               ERROR ("rrdtool: `RandomTimeout' must "
+                      "be greater then or equal to zero.\n");
+               }
+               else if (random_timeout==0) {random_timeout=1;}
+               else {random_timeout_mod = random_timeout * 2;}
+       }
        else
        {
                return (-1);