[FIXED] How to convert Java Hashmap Hierarchy to its equivalent in Redis Cache?

Issue

I have the following Job class representing runs of a job. I have a list of start and end times in Job class because the same Job can be rerun.

public class Job {

      private final List<Timestamp> jobStartTimes = new SortedList<>();
      private final List<Timestamp> jobEndTimes = new SortedList<>();
      private String jobName;
      private String jobKey;
      private String host;
      ....
      ....
}

I have this Map for querying jobs given jobkey.

public class JobMap {

         /**
         * Here value of 'String' key is jobKey
         */
         private final Map<String, Job> jobCache;
}

I have also created the following hierarchy of hashmaps for storing (starttime, jobKey) and (endtime, jobkey) entries in Map so that I can retrieve job records faster. This is needed because my queries are timestamp based, for ex: return all jobs that ran between x and y timestamp.

public class YearCache<T> {

        /**
         * Here value of 'Integer' key is month number (0, 11)
         */
        private final Map<Integer, MonthCache> monthCache;
}

public class MonthCache {

        /**
         * Here value of 'Integer' key is week number in a month(0, 4)
         */
        private final Map<Integer, WeekCache> weekCache;
}

public class WeekCache {

        /**
         * Here value of 'Integer' key is day number in a week (0, 6)
         */
        private final Map<Integer, DayCache> dayCache;
}

private class DayCache
{
        /**
         * Here value of 'Integer' key is hour value in a day (0, 23)
         * T is of type String representing jobKey
         */
         private final NavigableMap<Integer, NavigableMap<Timestamp, Set<T>>> hourCache;
}

I want to get rid of this Java hashmaps and move to Redis Cache. How can I model this hierarchy in Redis Cache?

Solution

This is needed because my queries are timestamp based, for ex: return all jobs that ran between x and y timestamp.

Take a look at Sorted Set Time Series Pattern in Redis Time Series Patterns. Details can be found here. You might have to tweak the example by a bit to suit your need. Excerpt from the doc:

+---------------+-------------+
| Timestamp     | Temperature |
+---------------+-------------+
| 1511533205001 | 21          |
+---------------+-------------+
| 1511533206001 | 22          |
+---------------+-------------+

You can query like this:

> ZADD temperature 1511533205001 21
(integer) 1
> ZADD temperature 1511533206001 22
(integer) 1
> ZRANGEBYSCORE temperature -inf +inf WITHSCORES
1) "22"
2) "1511533206001"
3) "21"
4) "1511533207001"

This is a simple example. The link discusses an extension to this approach to handle a specific corner case.

For your case, given that you have jobStartTime and jobEndTime, you might have to use separate sets for jobStarts and jobEnds keyed by the timestamp in which the job was started/ended. Then:

Let x denote the start and y denote the end of a job

1. Query all the jobs that were started after x from jobStarts set
2. Query all the jobs that completed before y but started after x from jobsEnds set
3. Repeated values from the results of step 1 and 2 yields the jobs that were started after x and ended before y

In your code:

public class Job {

      private final List<Timestamp> jobStartTimes = new SortedList<>();
      private final List<Timestamp> jobEndTimes = new SortedList<>();
      ....
      private String jobKey;
      ....
}

you're storing all the runs as part of the job. It's better to segregate the job runs separately from the job so that the job details can be cached separately in a set called jobs. Then the values of sets jobStarts and jobEnds can be the jobKey using which you can look up the details from the jobs set which stores the job details separately.

Answered By - kayvis

Answer Checked By - Pedro (PHPFixing Volunteer)