Statistics

All posts tagged Statistics

iTunes Missing Feature – Library Statistics – Revised

Posted by Ripdubski on 2020.12.06
Posted in: Programming. Tagged: , , , , , . 2 Comments

A while back (2018) I created a utility to display playback statistics from iTunes, now Apple Music which I called iStats (https://unfinishedbitness.info/2018/05/04/itunes-missing-feature-library-statistics).  Apple deprecated the automatic creation of the music library XML file not long ago, and that diminished my enthusiasm and support for my iStats utility.  Recently I discovered you can generate a compatible XML file with Apple Music so I added few things to iStats and cleaned it up a bit.  Though no longer automatic, you can still create it on the fly using menu item File > Library > Export Library…

During the update, the tools and resources I used:

One of the first updates I made was to clean up the Javascript code and start the process of code re-use.  The main effort to date is making functions to set the chart colors.  This cut down quite a bit of redundant code and there is still more work to do.  I’ll work on more later.

The next updates I wanted to include were:

  • Ability to set number of ranked results in charts.  There was a static 10 defined before, now it will allow the selection from 1 to 10.
  • Ability to export the song library to CSV file.  This will allow import into big data analysis tools like Splunk (https://splunk.com) and Elasticsearch (https://elastic.co).  With these tools one can perform all kinds of analysis and reporting on the library.
  • Ability to metricize song duration and total listening time broken down in various terms.

The main page is pretty much the same, along with new version information:

 

The primary difference on the main page is the addition of two more input elements.  These two elements should be set before selecting the music library XML file in the third element since that one starts the report generation process:

 

The elements are self explanatory.

  • For the results number, choose from 1 to 10.  The default is 10.
  • For the export, simply check or uncheck.  The default is checked.  If checked, a CSV file called AppleMusicLibrary.csv will be created and saved to your browsers default save location.  Some browsers may ask for permission to do this.  The song durations in the CSV file are for the song itself, not the duration of listen time for the song.  Everything else exported in it should be self explanatory.

Before starting, open Apple Music and export the Library file in XML format.  Select File, then Library, then Export Library.  Give it a name and location of your choosing.

To begin report generation after selecting the report options, use the “Choose file” button to open a native file selector.  Locate and select the library XML file you previously exported:

Once selected, the application then processes the file, and updates the Document Object Model (DOM – https://www.w3schools.com/js/js_htmldom.asp) with the graph data.  You won’t see any screen updates.  This is due to how Javascript works within the browser.  Any changes in the DOM are rendered in the browser window when the Javascript completes.

At completion, iStats will try to save a file called “AppleMusicLibrary.csv” to your browsers default save location.  You may have to give it permission.

I’ll refer you to the previous iStats post (https://unfinishedbitness.info/2018/05/04/itunes-missing-feature-library-statistics) for general overview of the statistics generated and section output.  

With that said, there is a new section which will appear before the Results section.  This new section is Total Time Listened.  It displays the total combined duration all songs have been listened to accounting for play counts of each.  It will display one line showing the total days, hours, minutes, seconds, and milliseconds.  Following that will be alternative views for total seconds, minutes, and hours listened as single entities:

 

To use iStats, simply extract the linked zip file.  To make things easy, a copy of chart.js, list.js, and filesaver.js are included (distributed via MIT License), but you should download them yourself.  For your own sanity, review the Javascript source in the “iStats.html” file so you can see there is no funny business going on.  It reads the XML file, updates the html page DOM, and optionally saves the csv file, thats it.

iStats is released under the Simplified BSD License.  Feel free to use, modify and improve.

Link to the iStats zip file containing everything you need (hosted at Google Drive): iStats-Distro.zip

iTunes Missing Feature – Library Statistics

Posted by Ripdubski on 2018.05.04
Posted in: Programming. Tagged: , , , , . 7 Comments

Version update at this new post: https://unfinishedbitness.info/2020/12/06/itunes-missing-feature-library-statistics-revised/

Love it or hate it.  Those are the only two terms I hear from people about iTunes.  This article isn’t going to debate the pro’s and con’s of using iTunes.  No matter which camp you are in, there is one undeniably missing feature – library statistics.  With statistics generated from your library, you can learn a few things about yourself.  I’ve used a couple of tools in the past, and even tried some different library management software that include some basic statistics.  None kept my attention for various reasons ranging from size, ease of use, Java requirements, or simply not enough detail. This article explores the implementation of a standalone statistics generator that meets my needs.

To create this application, which I have called iStats, I had only a few requirements:

  • Must be pleasing to look at (nice colorful graphs).
  • Must be quick (short run time).
  • Must be easy to invoke (as easy as launching a program).

With that in mind, I thought about writing it in several different languages.  In the end I decided that simpler was better, so why not write it in Javascript which can be encapsulated in a single html file.  This would allow double clicking the html file to launch it.  At the same time, I could brush up on my Javascript, learn about HTML5, and learn a charting library.

The tools and resources I used:

That most difficult part of the entire project was learning chart.js and configuring the graphs.  While the documentation is good, there is room for improvement in a few areas.  The CSS, Javascript, and HTML5 code are all enclosed in a single html file: “iStats.html”.  This file has a reference to pull in chart.js and list.js.

The application once launched:

Use the “Choose file” button to open a native file selector.  Locate and select the iTunes library XML file (“iTunes Music Library.xml” on a Mac):

Once selected, the application then processes the file, and updates the Document Object Model (DOM – https://www.w3schools.com/js/js_htmldom.asp) with the graph data.  You won’t see any screen updates.  This is due to how Javascript works within the browser.  Any changes in the DOM are rendered in the browser window when the Javascript completes.

Reading a file in the local file system from the browser via Javascript was a bit tricky.  This relies on the Javascript FileReader (https://developer.mozilla.org/en-US/docs/Web/API/FileReader) object.  If your browser doesn’t support the API call (or its disabled), then iStats won’t work. It uses the FileReader readAsText method to read the XML file, then parses it one line at a time.

Once processing is complete, about 10 seconds for my library of 6,700 songs, the page is rendered.  At the top are three list sections.  These were generated into the DOM using the “list.js” library.  The first (leftmost) column are the overall library statistics.  The second (middle) column are the statistics related to play counts.  The third (right) column are statistics related to skip counts.  The lists are ordered such that related items are on the same row:

The remainder of the page is consumed by graphs.  A few examples follow.  At the bottom of this article is a PDF of an entire iStats page where you can see all the graphs, and the iStats source code.

Notice the call out here.  Hovering over a bar on the graph shows the exact value:

This happens to be the last graph of the page.  It shows a timestamp of when the page was generated:

Example PDF of entire iStats output:

https://unfinishedbitness.files.wordpress.com/2018/05/istats-report.pdf

To use iStats, simply extract the linked zip file.  To make things easy, a copy of chart.js and list.js is included (distributed via MIT License), but you should download them yourself.  For your own sanity, review the Javascript source in the “iStats.html” file so you can see there is no funny business going on.  It reads the XML file, and updates the html page DOM, thats it.

The code could stand to be tightened up.  There is plenty of room for code re-use.  I got it to the point where it does what I want and works, but I haven’t cleaned it up.  iStats is being released under the Simplified BSD License.  Feel free to use, modify and improve.

Link to the zip file containing everything you need (hosted at Google Drive): iStats.zip

Version update at this new post: https://unfinishedbitness.info/2020/12/06/itunes-missing-feature-library-statistics-revised/