sdjson-cli

sdjson-cli

A CLI tool for interacting with the Schedules Direct JSON data service. This tool provides an efficient data fetcher that maintains a local cache of your listings data. Run the fetcher periodically to update the local cache and use the local cache as your EPG data soruce in your other applications.

Project Status: Beta

The current state of main will produce a db file that maintains a local cache of your SD account's EPG data. It's stable enough for me to continue on with my other project that's using this data, but if you browse the source code you'll likely notice that the unit testing is a little light. Some areas are well tested, others are not.

Development builds will soon be made available, including an initial v0.0.1 release. All v0.x releases should be considered usable, but beta quality. If you are using this tool, let me know. As I said above, this is stable enough for me to grab the data I need for the personal research project I'm working on that sent me down this road to begin with. If no one else is finding this useful, development will slow down to just bug fixes and other enhancements as needed for my other project.

How to Use

Setup

sdjsoncli --login <SD_Login> --password <SD_Pwd> status

Start with the status command just to make sure your login creds are valid. The tool will store your creds in the database file for furture use. You do not need to provide your login and password again unless you need to change it or if you delete the db file and therefore need to create a new one. This command (and any command) creates the sdjson.db file. This is the file that holds your locally cached EPG data. Use the --db-file option to use a different path and/or name for this file.

Find Lineups

sdjsoncli search lineups --country USA --postal-code 90210

Next, search for lineups to add to your account. EPG data can only be fetched for lineups that are registered to your account. Schedules Direct imposes limits on the number of lineups you may have registered and how often you can edit the lineups registered.

Note: Use the search countries command to get a valid country code for input into this command.

Add Lineup

sdjsoncli lineup add --id=[STRING]

Add a lineup to your account. The id value must be a valid lineup id as found via the search lineups command above. Remember that SD imposes limits on the number of lineups and how often you can modify your lineup registrations.

Fetch EPG Data

sdjsoncli fetch

Fetch all available EPG data for all registered lineups into the local db cache file. Run this command periodically to keep the local cache current. Successive calls only fetch data that is missing or has changed since the last call. Depending on your application, calling fetch once a day is usually sufficient. At most, 3-5 times a day. Excessive fetching could have your account flagged for abuse with Schedules Direct.

Other Commands

Use the -h option for a full listing of all available commands. But the ones shown above will get you your EPG data.

File Format for db

The db file is a standard SQLite database file. Use standard tools to connect to the file and explore the schema. The tables should be self evident as to what data they hold. One thing to note is that the database tables are not normalized nor do they contain columns for all fields of the SDJSON data. Instead, most tables have a payload column that contains the JSON encoding of each object. There are just enough columns alongside the payload to identify the contents of the payload (object ID, timestamps, etc.).

How Do I Use the Captured EPG Data?

The db file is the target output of this tool. Run the fetch command periodically to update the db. Once the db file is current, use the database as your local EPG source for whatever applications you need the data for. To use the data you'd usually connect to the SQLite db using a database driver for your platform. For example, you might use a SQLite JDBC driver to connect to the database in a Java application.

There are currently no plans to provide a client interface library/tool specifically for the database produced by this tool. If I were to release such a client interface, it would be for Go only as that's the language this tool is written in and the language I'm using for the other project that I'm using this data for.

The other thing worth noting is that the JSON objects stored in all payload columns found in the database are direct serializations of the data structures produced from the OpenAPI spec I've written. The REST client generated (sdjson.go) in this project was generated from the OpenAPI spec. If you wanted to use higher level language bindings for the payloads stored in the database, use the OpenAPI spec to generate the data types for you. For Go, I've been using oapi-generator to generate the client and data types. Alternatively, you can just parse the payload columns as JSON and use any JSON lib to work with the data. Ultimately, any app you write against this data will be a mix of SQL queries to find the data (lineups, schedules, stations, programs, etc.) you're looking for and a JSON parser to parse the JSON stored in the various payload columns.

Missing Features

Logos/Artwork

Station logos nor series/episode/celebrity artwork is downloaded/cached. I didn't need it for my underlying project, but more importantly, the local db didn't seem like an appropriate place to store this media. Stations do have the reference to their logos in the json payloads. A consumer of this data would then be responsible for fetching anything of interest. The other media is not stored in the db file. The infomation to fetch artwork is available in the program payloads. Programs include credits and credits have person ids that you could then use to fetch celebrity artwork. Programs also include the series id, which can then be used to fetch series/episode artwork. See the Schedules Direct API documentation for info on how to use these people/series ids to fetch artwork, logos, and other available media.

Various Endpoints Not Implemented

Only those endpoints needed to fetch the EPG data were implemented. Other endpoints for multi language options, some search options, etc. were not implemented. If you need them, let me know and maybe a future release will add them. Any endpoint to be added must first be added to the OpenAPI spec as the REST client used by this tool is generated from that spec. All the endpoints needed to maintain the EPG cache are implemented.