Wednesday, December 30, 2009

Version 0.8.2 released

You can check the change log for details or ask iTunesDSM to check for a new version for details, but essentially numerous fixes have been made.  If you install it in Windows, it will ask to uninstall the previous version first.

Sunday, December 27, 2009

This weeks' plans

First off, I had 57 downloads today!  I'm quite impressed.  Please remember though, I have put a lot of time into this.  If you find iTunesDSM useful please consider donating to the project.

For this week, I plan on finishing testing 0.8.2 which improves the iTunes communication.  It seems 0.8.1 has trouble starting the folder watching.  It should start with essentially an add new tracks task and then check for start monitoring for new folders, but the first add task doesn't seem to work consistently and it jumps straight to the folder watching.  The next version should have the fix for this.

There are a few other major updates I've talked about before, namely, adding new tracks to iTunes when files are copied to iTunes will be less likely to create duplicates.  Also tracks are added directly to the library and the process of adding is monitored with iTunes to make sure the process is completed.  Lastly, I plan on creating a playlist at the end of the add tracks process just to see the new tracks added.

I hope everyone is having a good holiday.

Brian

Thursday, December 24, 2009

A few more updates

Not much to say today, except that new tracks are now added directly to iTunes instead of through a playlist.  iTunesDSM will now wait until itunes is explicitally finished adding tracks using the ITOperationStatus object.

I spent a lot of time trying to rewrite some iTunes controller objects to make them more reliable but it was a lesson in huge frustration and I have since added to the iTunes controller that I had been using.

The next version will probably be available after Christmas.  Hope everyone has a great holiday.

Brian

Wednesday, December 23, 2009

Jacob can't co-create object exception

This exception comes up every now and then.  It sucks because you don't see it right away too.  Anyway a preliminary fix I have come up with is to make the object that utilizes Jacob a global variable.  That way the Jacob threads are only created once and the communication to the COM object is only instantiated once.  It seems to work for my purposes.  I'm sure something else will come up later...lol.

Tuesday, December 22, 2009

How to setup iTunes Folder Watching

As of version 0.8.0, iTunes Duplicate Song Manager can watch a folder for new files to add to iTunes automatically.  The folder, or music directory, is the same directory you have been using to search for duplicated files or files you want to add to iTunes manually.

So how does it work?  To get started.
  1. Select your options, the music directory, the file types you would like to filter(or add automatically in this case), and be sure the iTunes XML file was found successfully.  If not, you will have seen a warning.
  2. Enable background monitoring by checking "Monitor the music directory for new tracks and add them to iTunes in the background."
  3. Determine how often you would like to check for new tracks.  As of now, you can scan as often as every hour, or up to once every 6 hours.
  4. Press the "Start" button.
  5. That's it!  iTunesDSM with Folder Watch will run in the background and periodically check for changed folders.  If a folder has been changed, then that folder will be scanner for new tracks and if found will add them automagically to iTunes.

More fixes added today

Just a quick rundown of the fixes added today:
  1. iTunesDSM removes old playlists from adding tracks.
  2. Installer will remove old startup shortcuts.
  3. The FileChoosers all now show the iTunesDSM icon.
  4. Installer removes old version before new install
I'm still testing version 0.8.2 but will probably release it later this week.

Monday, December 21, 2009

Copying files and hard drive space

Due to the new anti-duplicating code, iTunesDSM must create a copy of the file before readding it to iTunes if conditions are right. This can get pretty spacey, so I've added a check to make sure the file space is available. Should have probably been there in the beginning but eh.

41 Downloads yesterday

Checking up on the sourceforge stats, there were 41 downloads yesterday (the highest it's been in one day)! That's a lot of de-duplicating! The program has been on sourceforge for about a month now and it's gotten close to 1000 downloads, which I think is pretty cool!

A couple fixes planned for the next version:
  1. Code has been added that helps prevent creating duplicates when auto adding tracks, though this now requires the user specifically enable iTunes keeping the directory organized.
  2. The installer/uninstaller needs to remove any startup shortcuts that may be present.
  3. Sometimes the xml parsing would fail if the iTunes music directory was a network drive on boot up. This has been fixed.
Brian

Saturday, December 19, 2009

Version 0.8.1 out

Fixes a minor issue in creating a startup shortcut in windows 32bit.

Version 0.8.0 Released - Automatic Background Folder Watching

I just uploaded the newest version : 0.8.0 with automatic folder watching/background monitoring.

I'm excited for this release because it allows the user to set and forget iTunesDSM to check a folder for new music and automatically add it to iTunes! Something my family surely needs with 4 of us adding music all the time, it's hard to keep our iTunes in sync.

Windows users - there is a new installer for you.
Mac users - it's the same as before - run the jar file.

I've uploaded some new screenshots and updated the release notes.

Brian

Friday, December 18, 2009

It's funny...

How you think you're done and then nope. You find something else.


A few fixes today. The iTunes XML files is still found automatically but the algorithm is much better. It searches through the entire user directory, if it finds one file - golden. If it finds more than one file, it will ask for the correct one. Hopefully this doesn't happen to the average user though. It only makes sense if someone is maintaining more than one library.

Up next is the usual Netbeans can't create a preview. It turns out my "global" logger objects passed around like usual objects was causing Netbeans to not be able to initialize gui builder previews. That one took a while to figure out.

Lastly, is a much improved Installer for windows. It performs an automatic check for Java using the JREDyna plugin for NSIS. Super cool!

Brian

Thursday, December 17, 2009

Switched to SysTray as the top level class

Only really for the techy people but I saw that iTunesDSM was taking a lot of memory, and so I switched the top level class to the system tray icon. I'm still testing it now.

Wednesday, December 16, 2009

Javadoc updated

I updated the javadoc today. It took forever, but it's part of releasing the code since it is open source. Version 0.8.0 is close to being ready. I've been using it on a regular basis on my system and its stable.

Tuesday, December 15, 2009

Creating an Exe file from java updated

Just a minor update for the previous post here. It looks like the NullSoftInstaller program works well for Installers...go figure. But creating a simple EXE file from a Java jar is better served through Launch4J. It's a simple program that can either create an exe file that runs the jar file. Or it can wrap the Jar file and create a standalone exe file. I've been choosing the wrap Jar file option to create a simpler exe file for users. This has the added benefit of seeing a real exe file in the Windows Task Monitor rather than another javaw.exe file. Either way, it definitely makes things better for users.

A few other UI updates on iTunesDSM as well. The system tray will only report new tracks that have been added if there truly were tracks added. It used to say tracks added based on a changed folder, even if no new tracks were added. Seems logical anyway.

Also checking for a new version will now report what the new version's big features are.

Finally, a few support options have been added to both the blog and to the program.

Lastly, got my final grades today. Straight A's for the last term. My final GPA was 3.67! Hope all is well.

Brian

Troubleshooting and FAQ

1. “iTunesDSM can find duplicate files but when I click the Clean iTunes button nothing happens.”

This can happen if you try to run iTunesDSM from within the zipped folder (.zip) you downloaded. Extract the zip folder and try running iTunesDSM again.

2. “Sometimes when I click the Add Tracks to iTunes button, nothing happens.”
This appears to be an issue with iTunes and possibly something called JACOB. Unfortunately, this issue is intermittent, and the only way to fix it is to close iTunesDSM and run it again.

3. "iTunesDSM is not finding any duplicated files, but I know they are there."
iTunesDSM relies on iTunes keeping your music files organized. The setting is in Edit > Preferences > Keep iTunes Media Folder organized. If this option is not enabled then iTunesDSM will not find any duplicate files.

4. "Can I search for duplicate files in other directories?"
Yes, just change the music directory for each search you want to perform and then rerun the Find Duplicates and/or Add New Tracks searches.

5. "What are exclusions?"
iTunesDSM has to keep a list of files that have already been added to iTunes from outside of the iTunes directory because adding tracks to iTunes programmatically can produce duplicated files. In other words, you can add files inside iTunes' music directory without incident, but adding songs outside of the directory can cause duplicate files to occur. It's all maintained automatically so you shouldn't need to worry about it.

6. “What happened to Mac support?”
iTunesDSM is written in Java so yes it does work on Mac, but iTunesDSM relies on COM’s to read information from the iTunes libraries. COM’s are not available on Mac so a lot of the iTunesDSM functionality is lost. I have looked into Apple Scripts, but since I don’t have a Mac, I can’t test them.
Be sure your Mac is set to use JRE 1.6 to run iTunesDSM. You can find this option in Java Preferences, search for it in your applications.

7. “What are orphaned files or orphaned tracks?”
Orphaned tracks are those tracks in your iTunes library that have the exclamation mark next to them. The exclamation mark means that the file that iTunes used to use to play that song no longer exists. iTunesDSM can automatically remove these orphaned tracks in the Remove Duplicates tab, while keeping the rest of your library intact.

8. Still having problems? Try the support forum.

Sunday, December 13, 2009

I graduated!!! And updates on 0.8.0

Background monitoring is still on the schedule. It's taken longer than I first imagined it would to square everything away. I've been switching over a lot of the code to use a Status interface class to make the classes more versatile should another developer decide to use them. It's also made it far easier to move everything to one status bar at the bottom of the program. The program still looks the same, so their is no need to post a new screenshot.

In other news, I had my commencement yesterday, so I am now a graduate degree owner!!! It was also my birthday and I am now 24. So, in quick successions, I graduated with a Mathematics degree in December of 2008 and now a Software Engineering Masters degree in December 2009 all right at 24 years of age! Now I need a job and health insurance : )

So if anyone knows of a job opening by all means let me know.

Thank you everyone who is reading and/or keeping up with the development of iTunesDSM and I plan on continuing development for as long as iTunes has issues.

Brian

Wednesday, December 9, 2009

Create an exe file with a Java application

Searching for this myself I see this comes up quite a bit and it can be expensive. I may have posted before on how to do this but I now use something called NullSoft Scriptable Installer. Basically, you write out some easy code, and if you reading this you probably understand how to code, and then NullSoft creates an Installer for you.

I know what you are thinking....I don't want to install it....or maybe you do....or maybe you need both like I did...I wanted both an installer and an exe file. It's possible with NullSoft and it's free.

The details are on their sourceforge page here. Hope this helps.

Brian

System Tray and More Background Monitoring...

I realized it's been a few days since I've updated the blog but the screenshots should make some of the improvements apparent.

Up first is the system tray. For background monitoring it's important that iTunesDSM be running but also not be intrucive and hence the system tray is used. The right click options are basic, a simple open and exit command.


Next is folder watching. Essentially what happens is a thread checks for modified folders within the music directory on a timed interval. An event listener is added to the BackgroundMonitor object which retrieves the top level folder that was modified. This folder is then run through the AddTracksTask. There is one issue that can occur; however, and that is the gap in folder's between program executions. For instance, a list of the folders is recognized when the program is running. iTunesDSM then shuts down and new tracks are added to the shared music folder. iTunesDSM would not recognize the new tracks in between executions. So, a couple of methods were added to the FolderWatcher class that stores all watched folders and their last modified times. If a last modified time is changed between folder executions then iTunesDSM will notice that and add tracks.

Next is the status bar. I felt a common status bar is more useful than seperate progress bars and status labels for each panel. It's a little neater I think, basically the screen shot shows how it works.

Finally, I've added a donation button to iTunesDSM. Again, I hate to be pushy but it's nice to see your work is appreciated and donations always help.

Sunday, December 6, 2009

Folder watching update

I worked on quite a few things today. Namely, I changed the options panel for folder watching. I didn't like the JSlider I had on there before, and also checking new files in terms of minutes isn't really necessary. You can now select a 1-6 hour period in which iTunesDSM will check for new tracks.

Secondly, I've figured out how to add System tray icons. This is necessary for background running again as no one wants to see iTunesDSM in the taskbar if it is in fact running in the background. Therefore, if background watching is enabled and the user closes the window, it is minimized to the task tray. Unfortunately, background monitoring is Windows only since it depends on the AddTracksTask class that is also Windows only.

With the Task tray icon, I've also added display notifications that display out of the tray. It's the usual balloon tips, but it's useful for quick notifications.

Finally, background monitoring also requires a common status bar. Therefore, the new status bar is a common jpanel object that displays current running processes as well as a dedicated background status label and a progress bar.

I would have posted a screenshot, but eh...tomorrow. It's not quite ready.

More to follow tomorrow.

Brian

First donation today!

A little excitement today, I can't help but post it here. I got a donation through paypal last night! I hate to be a pusher but if you find iTunesDSM useful please consider donating, its nice to see your work is appreciated.

Thanks
Brian

Saturday, December 5, 2009

Background Folder Watching screenshot




Here's a screenshot of what the options panel looks like with background monitoring. It's coming along, and it mostly seems to work. I'm sure there are still kinks to work out and only time figures those out. Otherwise there are still a few other major things to do, namely, I need to figure out to minimize iTunesDSM to the system tray, that way when it's backgrounding it, it won't be on the start menu. Enjoy the screenshot and your saturday.

Brian

Friday, December 4, 2009

Writing your own events not eventlisteners in Java


So I've never had to do this before and it certainly isn't taught in any Java classes. They never even approach writing your own events in java. I think they barely scratch the surface of using listeners, but I wanted to write my own Events that could occur and not event listeners. Well it's not as bad as you think. These two resources explain it pretty well.

http://72.5.124.102/thread.jspa?threadID=5185967&messageID=9725297 - Java forums thread
http://www.exampledepot.com/egs/java.util/CustEvent.html - The first example depot example I came across.

The java forums thread actually tells you how to receive events when a directory or file changes and a little modification made it easy to actually get the directory that was changed.

Essentially the trick is to create three classes, a class that extends the EventObject class, an interface that extends the EventListener class, and your own class that actually performs whatever you want to watch events for. The screenshot above shows a watched directory changed using the code based on the links above. The returned directory here has had a New Folder folder added and hence the aerosmith directory has changed.

Basically one step closer to automated folder watching.

Brian

Wednesday, December 2, 2009

Folder Watching

I've gone back to the folder watching idea. File notification still isn't built into java, that's the next release, but I've seen some online forum postings about monitoring the last modified property of a file so I've decided to go that route. My basic plan of attack is to keep a listing of all folders in a directory and their last modified long number. On a certain interval, the thread will scan all folders again and if a folder has been changed then get that folder and do whatever with it. 'Tis a bold plan.

Brian

I'm Done with classes!!!

I've turned in all of my assignments, I'm about to graduate!!!

Tuesday, December 1, 2009

Turned in the powerpoint

I've finished the project power point today and turned it in. With a lingering sinus infection, I'm sure I sound like little miss piggy, but I'm glad that's done. Lastly, I have to turn in the paper and the source code. After that, and once everything is ok'd I'll turn the code over to sourceforge and fully open source it. I have a couple ideas for folder watching that I want to try out, but I'm not going to add any new code this week, only after I turn it in.

Monday, November 30, 2009

Final draft of report finished today

Weighing in at a hefty 3500 words, I have finished the final draft of the report today. I'll probably read it again later, but for now, play time.

Brian

Sunday, November 29, 2009

MyTechGuide.org posting

Checking out my google analytics page showed my program was featured on mytechguide.org. Also a quick search on twitter showed my project was featured on a couple of other blogs as well as tweets.

That's really cool! I've come back from my Thanksgiving family visit and I'll be working on more documentation this week. All of the project materials are due a week from today, so tomorrow I plan on finishing the paper. Tuesday I want to work on the powerpoint and narrate it. The powerpoint only has to be 5-6 minutes long, so it will probably be a demonstration very similar to the early youtube video. After I finish those two things the project will be essentially done.

Hope everyone had a good break.

Brian

Tuesday, November 24, 2009

Softpedia and xml parsing

Awesome news! I didn't have anything to do with it, but apparently iTunesDSM is making the rounds. I got an email that SoftPedia had picked up my program and added it to their database! My guess is they saw it on SourceForge and added it there but still way cool!

The xml parsing thing came up because I noticed in one of their screenshots they that had the itunes xml file set an excel file, which of course shouldn't happen, haha! So I've just added some notification of a failed xml file parsing. That will make an appearance in the next release.

Hope everyone has an awesome Thanksgiving! I'll be traveling and if I work on the project, it will be work on the documentation.

Brian

Monday, November 23, 2009

Report documentation also updated

I'm still working on the report for the capstone project as well. It's a slow process as I generally hate writing. I know what you're thinking, but you keep us abreast of developments on this blog?! I know but it's easier to write here, then a report.

Javadoc updated

I've updated the javadoc today too and I'll posting it on SourceForge momentarily. There are a few more classes involved mostly listener classes and a few Gui related classes. For the inclined the javadoc is available to read.

Version 0.7.1 released

The new version has mostly logging features built in to help diagnose issues. There are a few other minor additions, most visible is a row selection counter on the remove duplicates panel. If you have updating at start up or go to the help menu and select check for updates, you'll be notified of the new release. Other changes are listed in the change log.

Brian

Friday, November 20, 2009

Sometimes frustration is an understatement

You never really know how much you appreciate tracing a bug with the Java exceptions until you mess around with Netbean's GUI editor and for some reason it can't render previews anymore. Sometimes....you can get it to show you a message about why it can't make a preview. For instance, you can try the little eye preview button in the builder and it will show the little red icon in the bottom right. But it still isn't easy to find the real problem.

Other than that, I worked more on the Logging system today. It still logs to an html file, but almost the entire program produces log reports now. I was able to add a couple features that make it really handy too. For instance a very, very similar exception stack trace is printed to the log file just like it is to the console by Java. That's very handy, you can see almost exactly what happened, just like you would see it on the console. I also added a couple of other items that are logged.

The Options class will log itself as will the new SysInfo class, that basically provides information on the host computer.

Finally, the html files themselves are formatted in such a way that certain logs will be color coded for easy spotting. All pretty neat.

I'll probably post the new version here soon. I have one user I've been conversing with on Youtube who seems to be having issues and without any kind of reference to go off of it can be difficult to solve the problem.

Subversion,

I use TortoiseSVN to maintain version control as I noted in the blog a few posts ago since SourceForge will host SVN. But, sometimes I mess it up, as happened today, fortunately, I have like 8 backups of code, so nothing was lost.

Have a good weekend everyone,
Brian

Thursday, November 19, 2009

Incorporating Logging

I've been working on adding the Logging function. It was previously in the code, but basically slapped together to try it. This time around its much better. I basically have a global java.util.Logger object and then all other classes will reference that object.

It was easy enough to write out the LogRecord object to a file, but everyone knows it sucks reading a text file, so I've moved to an HTMLFormatter object, that's still a work in progress though. I'm stopping for today, I'll work on it more tomorrow.

Toodles
Brian

Worked on documentation and test classes today

I've added some more to the report today, mostly describing the removing duplicates classes. I've also been working on the JUnit classes which have kind of been neglected in the last week.

I've seen some youtube posts, and I've seen some action on the SourceForge page. It's neat to see people using the program! I hope it's working for everyone.

Brian

iTunes show (exact) duplicates option

Something I discovered today in iTunes. If you go to File > Show Duplicates it will show you duplicates based on track tag information in iTunes....but....

If you hold down the shift key and then click File > Show Exact Duplicates iTunes will show you the files that are exact duplicates the same ones iTunesDSM removes.

Nevertheless, you still have to manually select and delete each file, if you want an automated process iTunesDSM will do this for you.

Brian

Wednesday, November 18, 2009

Documentation work today

In accordance with the Capstone requirements I worked on the final report today. Kind of lame, but necessary.

On a side note, while writing out the pseudocode for the duplicate files algorithm today, I found I could simplify the algorithm much more. So verison 0.7.1 now contains the much faster algorithm. It'll be a little while before I post the new version though.

Brian

Email down

I just found out why my email has been down for the past few days and it looks like its going to take a little longer to get back up again. If anyone has emailed me anything, it may have been lost but I'll post an update when it's working again.

Brian

UPDATE: It's working now. Turns out my domain name had expired.

Tuesday, November 17, 2009

Version 0.7.0 released

The newest version has been posted and is now hosted on Sourceforge. You can download it with the usual link to the right. The change log blog post has been updated as well as the screenshots.

Brian

Version update server changed

I've been switching over to Sourceforge because I'll be graduating soon and I'll lose the free file hosting the university was providing. Included in that was the file that is read to be sure the version of iTunesDSM was up to date. For future reference the version file is now stored on the Sourceforge server.

Brian

Sourceforge up!

I have basically got all enough of Sourceforge working to begin using it. I have subversion running, which is really nice, for anyone who thought about setting up their own version control software, its definitely easy to just use Sourceforge. As I stated before, the source won't be released yet as I can't use anything other than what I write until I've graduated, but at that point, I will release the source code.

The download links have been updated on the right. Have a good night everyone.

Brian

Version .7 nearing

I've been doing a lot of work on the code lately, aside from the design documentation, and now recently migrating to sourceforge. Yesterday I was finally able to condense down the number of times a progress bar runs through to just one time, rather than show a new progress bar every time a new portion of the background task completes. This was much needed for the user's sake, because, every time a progress bar repeats I believe faith is sort of lost in the progress bar. Why have a progress bar if it keeps repeating from 0 to 100%? Basically to get this working it was a simple math equation, but its much better for the user now.

I've also removed the iTunes version checking as the implementation was poor, this puts it back on the user to make sure iTunes is up to date, but if people are looking for a way to remove duplicates, then I'm sure they are as good about keeping software up to date as I usually am. So, for now, the version of iTunes isn't considered.

Next, I've removed the jAudioTagger libraries from iTunesDSM. It was neat to see the track tag information show up next to the songs in the table, but didn't really serve any purpose. On the plus side, not having to read track tag information is a real speed boost.

Next, I have figured out how to only have to read the options file once throughout each instance of the program. It seemed obvious to do that, but I wasn't sure how to implement it. Now in the very top most class a public static final instance of the Options class is created and set to what the OptionsReadTask class returns. Therefore, the options instance can be accessed from any class later. This again improves speed of the program and greatly reduces the number of times files are accessed in iTunesDSM.

Finally, I've run across a threading issue that I hadn't even thought of. The communication to iTunes is accomplished by JACOB, essentially adding COM support to Java. When I created a playlist in iTunes to add music tracks to, sometimes I couldn't remove the old playlist in time, to create a new one, and the AddTracksTask background task would just finish instantaneously. So for the time being, I've had to remove the functionality that removes old iTunesDSM playlists, and just continually add a new playlist every time tracks are added through iTunesDSM.

Look for the new update soon as well as information on how to download from Sourceforge!

Brian

Sourceforge

I've decided that I will release the code as open source software, still donation based, but I'll post the code. Here's the deal, however; I can't use any code created by the community while I am still in the Software Engineering program. So, once I graduate in a month here, I will then post the source, but until then I'll use SourceForge as a file host and version control.

Here's the sourceforge page I just setup, I've got some more work to do on it but here's the link....

http://sourceforge.net/projects/itunesdsm/

Friday, November 13, 2009

Design - imifcation - nestyness

For those who don't know me, I make up words like those in the title. It's kind of a catchy introduction as well as a WTF moment for anyone reading it, but seriously can you turn down a story that starts that way?!

Anyway, I've been splitting time between fixing bugs as well as documenting the iTunesDSM program in an official Software Design - iness way. I thought I would first start with UML Class diagrams. For those who don't know UML is the defacto way of diagramming software components otherwise known as Classes.

In order to start UML diagramming, I thought I might start with Visio 2007, but quickly learned Java is not really integrated into the UML support Visio already has. Google saves the day. I first found some stop gap solutions, but then lo and behold a plugin exists for Netbeans that creates UML specifications that are reversed engineered from java files.

Now I know, it's a bad to design software in your head, but I feel that if you can do it, and you are the only person working on it, it's a small project, there aren't to many dependencies involved, and the sky is still blue, it's sorta ok. So I've had to go back and actually diagram the classes using the Netbeans plugin. The picture is an example of an automatically generated UML class diagram for my iTunesDSM project.



Neat huh??!! It saves a lot of time especially since my design-in-head theory is vindicated by automated tasks. I won't go into much detail of how to read this diagram, but for those who do know how you can see the methods of my SongTable class.

Next up is combining the Class diagrams into a comprehensible hierarchy. I also automated Use Case diagrams can be generated by the Netbeans plugin. It may be easier to use Visio for that though, I'll find out.

Monday, November 9, 2009

Short update today as the TS Ida is on its way

Just posting a short update today for the title's reason. I worked on more class testing using Junit. It can be supremely frustrating, but I'm now trying to test some of the Background tasks. It seems there are two ways to really test these. One is to test the direct methods, which makes it easier, since you can treat just as another class you've written, and then you can test the SwingWorkers execution itself, which is pointless I think because it should achieve the same result assuming the SwingWorker is tested sufficiently. I'll leave it at that.

Brian

Sunday, November 8, 2009

DefaultTableModel, JTable, and SwingWorker

It seems this problem occurs intermittently but is definitely related to threading. When a user decides to delete files that are deemed duplicates, the files are deleted and the entry is removed from the JTable. The process of deleting files and removing them from the JTable is done in a SwingWorker task with the JTable object passed to the SwingWorker. To remove a row I have a simple method in my SongTable, which extends JTable, that searches for the correct row and then removes it. Sometimes though this error occurs with different array indexes:

Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 8 >= 8
at java.util.Vector.elementAt(Vector.java:427)

and you can tell it happens because of threading due to the fact that other output to the command line comes in between the entire exception trace on the command line. In other words, there is more to the exception that is printed but some of my output is printed between exception lines. I have yet to figure this one out. I've tried synchronizing the methods (which was already there) as well as calling the wait() method on other threads using a simple boolean called available that would go to false when something was operating on the SongTable. Still nothing.

My newest theory is it has something to do with the JTable working in the Event Dispatch Thread and the SwingWorker, essentially a thread, interfering but I'm stuck there. I'll keep searching for clues however. Until then, toodles.

Brian

UPDATE: 11/8/2009 3:27
I think I have fixed the issue. Here's what was going on. Basically I was trying to modify a component on the Event Dispatch Thread (EDT) which is where the GUI should run. The component was being modified by a different thread and I would essentially have the producer consumer problem. Here are a couple of resources to check out...

http://forums.sun.com/thread.jspa?threadID=790880

http://java.sun.com/j2se/1.4.2/docs/api/javax/swing/SwingUtilities.html#invokeLater%28java.lang.Runnable%29 - look into the invokeLater method

So in the end, and this may not be the way to do it, but I can now modify components in the GUI/EDT from other threads using the invokeLater method inside of my remove row method. Here's the code...



public void removeRow(final File fileToRemove){
java.awt.EventQueue.invokeLater(new Runnable() {
public void run() {
int fileNameColumn = FILE_COLUMN_NUM;
int rowCount = model.getRowCount();
for(int i=0; i< rowCount ; i++){
String selectedRowFilename = (String) model.getValueAt(i, fileNameColumn);
if(fileToRemove.getPath().equals(selectedRowFilename)){
System.out.println("Removing row " + i + " " + fileToRemove.getName());
assert i < model.getRowCount();
rowCount--;
model.removeRow(i);
break;
}
}
}
});
}

Saturday, November 7, 2009

Release Notes

iTunesDSM version 0.9.6
-Fixed an issue where the iTunes directory would not be found on boot if background monitoring was enabled
-Backup zip file is created of added temporary tracks in case iTunes does not copy files to the music directory

iTunesDSM version 0.9.5 (2/4/2010)
-Fixed issue where new tracks would not be displayed in the song table when adding them manually
-New Options panel ui
-Multiple directories can be monitored, added from, or searched for duplicates in
-Files to add to iTunes are ordered so that files within the iTunes music directory are added first
-Options class now extends the Properties class has been converted fully to a Properties object
-Removed OptionsPanel class
-Options class checks the options file is a compatible version of iTunesDSM


iTunesDSM version 0.9.1 (1/26/2010)
-ProgressMonitorDialog box now has native look and feel
-ProgressMonitorDialog has overloaded methods for calculating percent complete
-CopyFile now has option to monitor progress as well as methods to verify copied file is accurate
-Cleaner SysTrayMain code
-Added OSUtils class
-ITunesXML class has more JUnit Tests
-iTunesXML class is logged to a seperate file
-AddTracksTaskTest class tests more fully
-Fixed System Tray menu for Mac - not allow background monitoring
-iTunesDSM Added Tracks playlist reinstituted
-Version class now uses class level logger
-Changed added tracks file system tray message filepath more readable
-Removed FolderWatcher classes, events, and listeners
-Removed AllFiles classes, events, and listeners
-Fixed Icon resolutions - Added Icon class
-System Tray Icon now retrieved through Icon class
-Next background monitor scan time displayed in System Tray tooltip


iTunesDSM version 0.9.0 (1/20/2010)
-Improved background monitoring reliability
-Added option to set music directory to iTunes music directory
-Added Background Monitor Events and Listeners
-Added start and stop background monitoring to system tray
-Cleaned notifications of background monitoring to system tray only
-Changed all event types to integers instead of strings
-User can now cancel the automatic iTunes XML file search
-Updated program titles
-Only one iTunesDSM window can be opened
-SysTrayMain class implements Status interface
-Log files are deleted at each execution
-Removed ITDSMLogger class
-Background Monitor status is monitored through SysTray icon tooltip
-Donate panel tool tip displays donate url
-Removed status bar
-Parsing of iTunesXML file locations by line now works
-Memory issues fixed
-Removed iTunesDSM playlist in favor of added_tracks.txt file
-Fixed a null pointer issue when the background monitor would try to start and the music directory was null
-Removed iTunesController package in favor of limewire iTunes code
-Options file saved and read from properties xml file
-Temporary iTunes download directories are ignored
-iTunesXML file search progress is displayed in the options panel


iTunesDSM version 0.8.2 (12/30/2009)
-Added check to see if iTunesDSM.exe is running before installing over an old installation
-Installer automatically removes previous version
-Added code to help prevent duplicates from being created when adding tracks(copies to a temp folder)
-Fixed iTunes music directory not parsed on boot when itunes directory is a network drive
-CopyFile class now warns of not enough disk space when copying file
-iTunesDSM now adds songs directly to the library instead of a playlist
-After all tracks have been added a playlist is created with all new tracks
-File choosers now show iTunesDSM icon
-Added seperate logging for Folder Watching
-Added ITOperationStatus to AddTracksTask to be sure iTunes is done copying files
-Added AddToiTunesLibrary class
-Fixed Working Directory in Startup Shortcut

iTunesDSM version 0.8.1 (12/19/2009)
-Fixed Windows 32bit startup shortcut creation

iTunesDSM version 0.8.0 (12/19/2009)
-Added background folder watching
-Added FolderWatch, AllFiles, BackgroundMonitor and Event classes
-Updated Options class to maintain booleans and time delays for folder watching
-Updated Options gui object for background monitoring
-Added SysTray class, TimeSpinner class, StatusBar, StatusObject class
-Added StatusBar
-Added AllFilesProgressEvent/Listener classes
-Removed ProgressMonitor class
-Simplified OptionsPanel code
-Simplified logging
-Improved automatic xml file finding algorithm and background processing
-Added Windows Installer
-Released Source code - I had to wait until I graduated.

iTunesDSM version 0.7.2
-Fixed a bug that would not warn of a failed xml file parsing.

iTunesDSM version 0.7.1 (11/23/2009)
-Simplified search for duplicate files algorithm in DuplicateFiles class
-Added rudimentary logging
-Added HTMLFormatter class, ITDSMLogger class
-Fixed a bug that would bring up iTunes when cleaning folder, but iTunes wasn't needed
-Added selected row count to remove tracks panel, RowCountLabel class, RowCountListener class, SelectionListener class
-Added SysInfo Class
-Fixed a bug that would sometimes cause Netbeans to fail at showing a preview of the GUI - propertyChangeListener on the update checkbox
-Added a dialog box that will warn the user if no options are selected on the Remove Duplicates Panel
-Removed RemoveOrphanedTask class

iTunesDSM version 0.7.0 (11/17/2009)
-Converted Options input and output to ObjectOut/inPutStream writers and readers
-Added OptionsReadTaskTest and OptionsWriteTaskTest classes
-Combined CleanDirectoryTask and RemoveOrphanedTask into CleanITunesTask
-Fixed a bug that would not correctly select all songs in the table if two or more filenames were exactly the same
-Removed jAudioTagger functionality resulting in much improved speed
-Simplified file data that is diplayed in the SongTable JTable by removing track title, track artist and track album information from the rows
-Fixed a bug that would not let the user know an unsuccessful update check was performed
-Added option to check for new version of iTunesDSM at startup
-Removed iTunes version checking
-Removed MediaFiles, FolderWatch and DataIO classes
-Simplified reading options file to one read per application instance
-Condensed AddTracksPanel and RemoveTracksPanel progress bar to one full progress bar
-Progress Bars now show percentage complete.
-Changed how the iTunes playlist is created to avoid threading issues, old iTunesDSM playlists are not deleted anymore
-Modified status labels that appear before and after tasks complete.

iTunesDSM version 0.6.3 (11/6/2009)
-Added Data class
-Added DataTest class
-Changed code that creates new options and exclusions data files

iTunesDSM version 0.6.2 (11/6/2009)
-Added major version number to current version
-Added Version parsing.
-Added iTunes version checking to ensure the proper iTunes version is installed.
-Removed indeterminate progress frame from ITunesXML class.

iTunes DSM version .6.1 (11/4/2009)
-Removed RemoveTrackDisplay class
-Minor UI improvements when scanning for orphaned files
-Added (.mov) video format extension
-Added JUnit tests for more core package classes
-Renamed AudioFileFilter class to MediaFileFilter
-Removed ITunesXMLTask class
-Removed the FileTypePanel class
-Minor source code improvements
-Fixed a bug that would not set the iTunes music directory to the correct directory after choosing another iTunes XML file.  The options panel is then updated with the new information.
-Fixed a UI bug that incorrectly displayed the progress bar in the RemoveOrphanedTask class.
-Finished first drafts of Javadoc for all classes.


iTunes DSM version .6 (11/3/2009)
-Remove Apache libraries that were used to read Property List xml files
-Added rudimentary new version checking
-Added JUnit tests for Options and DuplicateFiles classes
-Changed core.FileWriter class to core.CopyFile to keep java.io.FileWriter clashing
-Removed main class in core package
-Fixed a bug that would only find up to 8 duplicated files, will now find all duplicate files no matter the iteration number

iTunes DSM version .5.1 (11/2/2009)
-Added tooltip to custom extensions option.
-Fixed bug that displayed incorrect string representation of custom extensions in options panel.
-Added junit test class to Extensions class.
-Added official ChangeLog text file.

iTunesDSM version .5 (11/2/2009)
-Added safely add new tracks to iTunes
-Added options panel
-Added exclusions
-Added options input and output

iTunesDSM version .1
-First release

Version 0.6.3 Released

Mostly bug fixes. You can download it on the right.

Brian

Friday, November 6, 2009

Extensive work today

Basically I did a couple of things. I updated the Versions class to be much more useful. At first all it simply did was check if strings were equal essentially. And all I was checking was if a new version of iTunesDSM was being hosted. The Versions class is now a more full fledged class. Firstly, I now label iTunesDSM versions with the three integers pertaining to Major Version, Minor Version, and Bug fix version. I'm not sure if Bug fix version is the official term but it works for me.

Secondly, if I'm going to check for new iTunesDSM versions I might as well check for the version of iTunes itself, as any version below 9.0.2 will create duplicates more easily than the current version. In order to get the version of iTunes, I read the XML file, this required some additional code to the ITunesXML class and a few reworks of the methods. I now have a method that extracts a the value of an element given the line to look for. So I can basically search for iTunes specific items in the iTunes XML file. In order to check the version numbers I now parse this string for the three digits giving the version. Basically the string "9.0.2" will give the integers 9, 0, 2 and in one background task I now check for the latest version of iTunesDSM as well as the version of iTunes that is installed, and if iTunes is out of date, I warn the user before closing the program. I consider the newest version of iTunes as a necessity simply because you can't keep old versions of iTunes practically and therefore test them easily.

I also added a Data class today, to simply create a new "data" folder for the files that save user options and exclusions. It's got one method which should be really easy to test tomorrow.

Finally, I've updated the ReadMe document. It had a few grammar errors and some inconsistencies. Also as I've added/removed code, I've tried to keep the documentation up to date.

I'm working on version 0.6.3 as of now, so expect an update here soon.

Have a good weekend everyone.
Brian

Thursday, November 5, 2009

iTunesDSM Screenshots



Options tab. Here you set your options such as the directories you want to search for duplicates in or that you would like to add tracks to iTunes from, background monitoring/folder watching, file types to filter, and startup options.  Just click the item on the left to display those sub-options.



Here you can see that iTunesDSM Folder Watching / Background monitoring has been enabled and the user is notified via the System tray.



The Add tracks tab. Here you can have iTunesDSM scan for new tracks to add to iTunes from the music directory manually.  You can also click the View All Added Files button to view a listing of files added to iTunes.  You can also have iTunesDSM check for new tracks in the background essentially monitoring the music directory.



When you are ready to remove duplicate files you move to the Remove Duplicates tab and click Find Duplicates. Once the task is completed you select all or some of the files in the list and then choose how you want to process them. You can back up the files to another folder, and then delete the duplicates. Lastly you can clikc the Remove Orphaned Tracks button to remove orphaned files.  Orphaned tracks are tracks that have been deleted from the music folder but that iTunes thinks still exists. You can see this when the exclamation mark appears next to a track in iTunes. Use this feature to remove those tracks from iTunes.

First draft of Javadoc completed

It took a while and it's pretty mundane, but as every programmer knows, you have to document your code. So today I finished the draft of the javadoc. If you are interested you can view it here.

Wednesday, November 4, 2009

Documenting

Anyone want to raise their hand at the worst part about programming??!! Yep you guessed it, going back and documenting all of the classes you've written. Following a close second is going back and fixing bugs you find after you've run more tests.

What I've been working on today.
Basically the above. I've created some more JUnit test classes and I'm going back and documenting my classes as well. Overall I haven't found any new bugs to speak of, maybe minor things but nothing show stopping.

I've checking on the web stats for my blog. It looks like I get around 20 new visitors a day! That's exciting, remember if you find the program useful, think about donating to me. I can always use a cup of coffee. No amount is to small.

Hope everyone has a good day
Brian

Tuesday, November 3, 2009

Folder watching and JDK 7

I was interested in adding Folder Watching to iTunesDSM so that you could basically have it run at startup and then just automatically add songs to iTunes as new songs were added to the directory. I thought I found a way to do this too here but then I realized you need to run java 1.7/jdk 7 which is still in early release. I then though about implementing my own file watching thread thingy, but that's not really a good idea either as continuing walking the entire file tree every 15 secs is taxing to the system. I could set it up to scan every half an hour or something but even then, it would take a lot of resources. I'll consider it though.

Brian

Version .6 out

Updated version today, version .5 was shortlived but that's ok. .6 mostly fixes bugs behind the scenes. iTunesDSM would be unable to find duplicate files that iterations above 8, so if for some reason you had more than 9 duplicate files in the directory then iTunesDSM wouldn't remove all of them. That's now fixed. I also added a version check into the software. On startup it will poll my webspace for the most current version. If the versions are off, then you are given an option to download the newest version. Neat.
Brian

Monday, November 2, 2009

Working with JUnit

Today, after releasing the next big update, I began some formal Unit testing. Unit testing simply means that testing units or pieces of the program for bugs. I started with the Extensions class since that is the most direct user class. It's the class that handles parsing the custom extensions essentially. JUnit is interesting, it's integrated into Netbeans, though I accidently imported a newer version of JUnit than what Netbeans shipped with and it seemed to cause some errors later. No biggie now as it's just imported as any other Jar file. Not a whole lot to say about JUnit yet, other than the nice integration though. I plan on testing further this week.

I also got an idea for a very simple way to check for new versions from the client. I was thinking of a simply URL class that reads a text file on my server space from school. The same server that currently hosts iTunesDSM now. Should be simple. For now, toodles.

Brian

v.5.1 Released

Fixed a couple of bugs. Added a changelog file. Enjoy!

Brian

v.5 Released!!!

I'm posting the new version today. Here's what's new:
  1. Options panel that you can set your music directory and file types to filter.
  2. Add tracks to iTunes that exist in your music directory but not yet in your iTunes library. For example, users are sharing a directory for all their music and one user has bought a new song, but the other iTunes libraries don't know about it. Other wise known as folder watching.
I've added a readme file as well to help new users get up and running. I'm debating on whether or not to record a new YouTube video, I'm not really sure it's necessary, but I might do it anyway.

All of the features are supported on Windows, but unfortunately iTunesDSM relies on COM's to communicate with iTunes and therefore are not available on Macs. You can still use iTunesDSM to backup and/or remove duplicate files from a directory.

If you find any bugs, don't hesitate to email me or post a comment, I will try to fix them.

Enjoy!
Brian

Saturday, October 31, 2009

Another reason Apple's XML is pointless

A few posts ago I talked about how to read one of Apple's Plist XML files. Sure it works, but I have another reason to despise it. Not only is it tough to read in your own parser, not only is it tough to find a third party parser that will read them, it now sucks the memory of your computer too. Using the Apache parsers I found before I was able to read the directory iTunes stored music to, but it wasn't until I tried v.5 on other computers that I was getting memory heap errors, or basically the virtual machine was running out of memory. It took a few hours to pin the error down to reading the XML file. I watched the amount of memory that was used by iTunesDSM before and after reading the XML file. The memory used would be 50mb or so and then balloon to 300ish mb. Ridiculous. Of course that's not to say that other XML files wouldn't produce the same amount of memory requirements, the XML file itself is somewhere around 12 megabytes. But first impressions last, and I will always try to avoid Property List XML files.

I did fix it though. I strictly read the file for what it is now, instead of using a parser. I read the lines coming from the xml file and scan for the string that contains the music folder references. Not only is it much, much, much faster, it takes a lot less memory.

I also made a few changes to the FileWriter class. Instead of the CleanDirectoryTask coming up with files that don't already exist, the FileWriter does. I also made a few changes to the dialog that appears when copying files. As before a simple progress dialog appears when copying files takes longer than a set amount time. It's a neat way of doing it...and it's in the ProgressMonitorInputStream class.

Friday, October 30, 2009

What I worked on today

After figuring out specifically when iTunes makes duplicates at least as far as I can tell. I was able to remove the feature that would temporarily move a new track from the itunes directory to a temp folder and then re-add the file to iTunes when iTunes was set to copy music to the folder. iTunes seems to keep from creating duplicates this way, so it's not needed anymore. I've also removed the requisite options.

Exclusions are still in though, this helps keep iTunesDSM from re-adding files outside of the library and thus preventing duplicates since one of the ways duplicates occur is by programmatically adding new tracks.

Also, I've been trying to keep Mac functionality and one area where this was a problem was in automatically locating the iTunes XML file. Well using System.getProperty("user.home") I am now able to automatically locate the xml file without user intervention on Mac. If the user has installed iTunes elsewhere and iTunesDSM can't find the xml file, then it asks the user. Another mac problem that seemed to come up but turned out to be a non issue, is if the options file was open in mac, a Concurrency Exception was thrown by my ReadOptionsTask class. Turns out you can't have the file open in Mac, but Windows it's ok. I have a feeling Windows is using a temporary file there.

Next, when iTunesDSM opens the first thing it does is look for the xml file, so sometimes, it looks like it's taking a while to locate it and the gui would drag. I've now added an indeterminate progress bar to let the user know it's finding it.

I've added a couple things to the blog. Mainly some personal information about me. I also added a features list on the side, I hadn't realized that was missing but yet so vital. It's about to get a lot bigger too!

Have a good friday everyone.
Brian

When does iTunes make duplicates???

Ok I've done some tests and here's when it happens.
  1. Tracks are added to the Automatically Add to iTunes folder and the files already exist.
  2. You have reinstalled iTunes and you set the copy to itunes folder option on, and then add music from the same folder that iTunes is set to store music. Every file is duplicated.
  3. Adding songs programmatically. If you are adding tracks from outside the iTunes library and iTunes already has those songs in the folder (and the copy option is on) duplicates will be created.

Thursday, October 29, 2009

Couple more updates

It's kind of weird to figure out exactly when iTunes will create duplicates. So I've added an exclusions list. Basically, it's just a text file that contains the paths of files that have been added to iTunes through iTunesDSM. The usual SwingWorker classes have been added too. I'm done for today.
Brian

v.5 Updates and Screenshots






A few updates to talk about today...

I have been working on seperating out the logic. As noted before, the SwingWorker Tasks I had were bloated because I kept adding on top of them, especially the task that operates on iTunes. Now the itunes task has been split up into an AddTracksTask class and a RemoveTracksTask class. These two tasks now handle communication to and from iTunes as well as a smaller ITunesWin class that does the simple things like getting and creating the playlists.

Also I've done some work on the options panel. Like the other day it will load all of the options from a previous instance of the program. It didn't remember the extensions that were selected from the previous time, but now it will. Basically every option that is listed in the options panel is now reloaded from previous executions. I've also simplified the act of getting the extensions from the options object, it now directly returns the extensions array and not the Extensions object which you then get the extensions array from.

Also, in the options class is the isReady() method that checks to make sure all the options are set that would be required for other tasks to run. For example, checking for duplicates before setting a directory will now throw up a dialog telling the user to set options. This makes it much easier when I'm about to start a SwingWorker task as all the settings are checked in one place rather than before each execution.

I realized I haven't posted any screen shots in a while so I added a few to this posting.

Brian

Approval approved

I got my topic idea grade back, it looks like I get to keep developing iTunesDSM! I plan on recording the next update's youtube video today. And mayyyybbbeeee posting the next version here soon. It's already been termed version .5.

Wednesday, October 28, 2009

The official topic idea I submitted today

Posted here for consistency...

I’ve been referring to my program as iTunes DSM (Duplicate Song Manager) and have been keeping up with the blog that acts as the design diary. In the past few weeks, I have added the ability to add songs to iTunes in a careful manner to prevent iTunes from duplicating files. What I have in mind for this semester is to move beyond the proof of concept phase and start fully testing the program as well as documenting it. I would like to use the techniques we learned in the SE Testing class as well as learn JUnit to gain more confidence in the program functioning correctly. I also need to design more classes to distribute the logic. One of the Java techniques I have learned is implementing SwingWorker classes that act as background tasks for Java Gui’s but the side effect has been a lot of code in just three classes. Testing the program will help me divide up the logic more thoroughly.

Coincidentally, quite a few people have found my program useful, including my family so I hope to move it to a more robust state.

iTunes duplication and version .5

I've spent a considerable amount of time in an attempt to prevent iTunes from creating duplicates when you add songs that were in the iTunes music directory but not in the library. iTunes used to create duplicates in this situation, but now it seems it's much smarter. In testing, I have found that regardless of the option in iTunes that copies tracks to the directory, iTunes will not make duplicates. So this means, if the option is on and files are already in the iTunes directory, they will not be duplicated when added. It seems iTunes is slowly whittling away the errors that would create these duplicates.

It also seems that iTunes is much more careful about creating duplicates in the auto add folder. When version 9 first came out, it was more difficult to add duplicates, but you could still force it if you added songs to the automatically add to itunes folder. Now itunes will still create duplicates but not add them to the library. So you might not notice duplicates are there, but it is still much less likely you'll accidently duplicate tracks.

So, I'm trying decide on whether to keep the option that fixes the old copy files bug. I know for the most part everyone is pretty much using the newest iTunes. In a sense, you are forced too.

Anyway, I'm still working. I'll probably record a new video here in the next few days for the next version.

Brian


UPDATE:
Ok it seems, in the off chance there is still a possibility to duplicate songs that are added. Here's what happens. Suppose and album is already in the itunes directory but not in iTunes. Suppose now I add that same album outside the itunes directory on accident programmatically or by hand. iTunes will create duplicates and reference the duplicated files. Bad.

Again it's an off chance of something happening, but it means my hard work on keeping iTunes from doing this is a good thing.

Tuesday, October 27, 2009

How I have to handle options and jpanels

So I've been messing around with how to handle user options. It's amazing how much you learn from just doing. I have an Options object that I initialized in my top level object, the main class. I first tried to pass this object to all the objects created underneath i.e. my Panels. That didn't work because apparently JPanel constructors have to take zero arguments. You could tell that some kind of error occurred when Netbeans couldn't render a live view in the Design gui builder tab. So next I tried creating zero argument panels that right after have their options objects were initialized, I set the options object using a Setter method. This sometimes worked and sometimes didn't. It was really frustrating when sometimes Netbeans was able to render a preview and other times it couldn't. Netbeans is subtle about errors that appear by showing a little red circle with a white line icon at the bottom right of the screen. It took me a while to notice this since it would disappear quickly.

So how do you handle user options that must be synchronized between different objects when you can't pass the object in the constructor? I'm not sure how other programs do it. I'm guessing this isn't just a Java thing. But what I worked on today was a completely different way of handling options between objects. I basically, write the options to a text file and then read the file over and over again.

This presented a different problem though. It is now necessary to read the Plist xml file that iTunes maintains so that iTunesDSM and iTunes do not create duplicates. Reading the file takes a while. Back to my old friend SwingWorker. It's actually pretty easy to use SwingWorker objects. Later I'll post some examples. These have the distinct advantage that they don't freeze a GUI when longer tasks are executed. Basically the task is run in the background and the GUI still interacts with the user.

Now what basically happens is, as soon as an option is selected a SwingWorker object is instantiated to write the new options to the file in the background. When options are accessed later another SwingWorker object reads the file and returns the option object.

Tomorrow I need to read the Options object and set the Options Panel to the settings read in the file. That way, you set the options and they stay. Also, tomorrow the official topic idea is due. For formalization sake, I'll post the topic idea on here as well. The topic is supposed to be about what the final Capstone project will be and I want to use this project as my Capstone idea. If I am able to, then I'll push ahead with formal testing of iTunesDSM.

Sunday, October 25, 2009

Options and Settings

Making headway. I'm realizing how much of an update this is really going to be. I now have an options object that will keep track of user options and eventually I want to output these options to a file so that they don't have to be reset all the time, but for now they will reset when the program is re-run. I'm also consolidating the File types to filter options and placing that on the settings page as well. I haven't really decided how that will happen though. Also I'm noticing how large the SwingWorker task for ITunes functionality is gettings. Unfortunately, I'm not really sure how to fix that. It's not really good coding practice but I don't see how to make it any better as of yet. More to come this week.

In another note, I noticed I have a follower of my blog! That's cool. Hope everyone has a good rest of the weekend.

Brian

Saturday, October 24, 2009

Watched folder adding now keeps iTunes from creating duplicates

Short update today. Yesterday, I got iTunesDSM to correctly identify media files that were already present in the music folder and now keeps iTunes from creating duplicate files from those. I had to use the previous post's solution on how to read XML property list files to learn the location that iTunes stores music. The basic trick is to temporarily move the media file to an outside folder and then add the file to iTunes from that location. This is dependent on whether iTunes copies files to the media folder or not. There is still work to do as the number of options that are available are mounting. I have been debating on creating an Options object that just maintains all options the user would use. If I do then a new JPanel needs to be created that will list all the available options. I guess I got my work cut out for me.

Brian

Friday, October 23, 2009

Read a property list XML file in Java

Plist for short, a property list XML file is used by the Mac Operating System to store information about a whole host of applications for Mac. It's even used by iTunes on Windows as a secondary interface for information about all the tracks and playlists in iTunes. I have been searching forever on how to read these files without having to design my own Java classes as Plist files are simply pointless. My research report was on XML and Plist xml files simply destroy the XML ideal in my opinion. Regardless, someone at Cupertino thought they were a good idea. I finally figured out how to read them using Apache Commons java packages.

I jotted this down in my notes for iTunes DSM but you basically need to download these packages:
  1. Apache Commons Configuration - Plist XML files
  2. http://commons.apache.org/configuration/
  3. Provides Exceptions for Apache
  4. http://commons.apache.org/lang/
  5. Apache Logging
  6. http://commons.apache.org/downloads/download_logging.cgi
  7. Apache Predicate
  8. http://commons.apache.org/collections/
  9. Apache Base64
  10. http://commons.apache.org/codec/
The notes are just for me to know exactly what classes are required by the URL's listed there will take you to the web pages to download the packages. I usually just download the Jar files as it is easier later to simply replace one file should an update be needed. Once these packages are imported I use this code to get the location iTunes is storing music in.


import java.io.File;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.plist.XMLPropertyListConfiguration;




public class ITunesXML {

private static String SPACE_CHAR = "%20";
private static String LOCATION_PREFIX = "file://localhost";

private XMLPropertyListConfiguration config;

public ITunesXML(String location){
try{
config = new XMLPropertyListConfiguration(location);
}
catch(ConfigurationException e){
System.out.println("Exception ITunesXML Constructor");
}

}

public String getITunesDirectory(){
String loc = config.getString("Music Folder");
assert loc != null;
assert !loc.equals("");
loc = loc.replaceAll(SPACE_CHAR, " ");
loc = loc.replaceAll(LOCATION_PREFIX, "");
try{
File file = new File(loc);
assert file.exists();
}
catch(Exception e){
System.out.println("Error in getITunesDirectory");
}
return loc;
}

public static void main(String[] args){
ITunesXML xml = new ITunesXML("C:\\location\\to\\iTunes Music Library.xml");
System.out.println(xml.getITunesDirectory());

}

}


So here's a simple example on how to read the Mac OSX property list xml files in Java using Apache Commons packages! Now on to code the rest of the bug fixes.

Brian

For new users....

I realized this morning I have a lot of postings about things that don't work. These are all references to the next version (.2). The version you can download now (.1) is still good to go. It will still remove duplicates from the iTunes folder and clean iTunes of orphaned files. You can download version .1 on the right side of the blog.

This blog is used to fulfill a requirement for my Software Engineering masters project and so this is acting as a diary of the design/build process.

On another note, I woke up this morning with a solution to the previous posting about duplicates being created by the Add Tracks process if iTunes is set to copy songs to the folder. As of now, the solution depends on asking the user if the Copy option is selected in iTunes and if it is, then iTunesDSM will make a temporary copy of the track to an outside directory and then import that file into iTunes simultaneously deleting the file that was already in the iTunes folder. Basically, copy file, delete file, import file, iTunes copies file. If iTunes does not copy files to the music folder, then the file will remain in place.

I will try and code this solution this afternoon.

Thursday, October 22, 2009

Found a bug today


I've been trumping up the ability to add songs to itunes that are in the directory but not in the itunes library, for example, the file exists in c:\music where itunes stores the music, but the file has not been added to itunes yet. The bug occurs when the "Copy files to iTunes Media Folder when adding to library" option is turned on. Here's what happens....
  1. Add songs to library using iTunesDSM...itunes adds the song to the library and adds the file to the organized folder in the library root folder
  2. If the file was already there, it is duplicated, i.e. a number is added to the end of the file.
  3. iTunes references the duplicated file in the library...not the original.
  4. Tab over to the remove duplicates tab in iTunesDSM and the program dutifully finds all the recently created duplicates and the user backs them up, deletes them, and removes orphaned files in iTunes.
  5. All of the songs that were added in step 1-3 are removed since they are orphaned and we are now back to where we started.
How to fix this in the short term is to simply turn off the "Copy files...." option in iTunes while the adding process is progress assuming the files you want to add were already properly organized by another user.

If they are not properly organized before hand, the files will remain in place and itunes will reference that song outside of the usual itunes library.

This will be a tricky fix and will delay the next release.

In other aspects of the program...I have created a new icon for the program (at the top). It's basic but it gets rid of the java cup icon. Also the backup function will ask the user should the duplicate file overwrite an existing file. It's unlikely to happen but it needs to be there.

Brian

Wednesday, October 21, 2009

Still Working...

Still working. I keep messing around with different ideas I get. iTunes DSM is mostly ready for a next release but I like to tinker. You can still download version .1, but .2 will definitely have manual folder watching.

I have been thinking about how to run folder watching in the back ground, but the more I add, the more restricted the program becomes. For instance, the folder watching now only works in Windows as do most of the other features, which are disabled at start up on a Mac.

Some other more official blog notes...
Even though I wrote the MediaFiles class to find the files under the music directory, I can't use the class as SwingWorker classes allow you to cancel a task but that cancel boolean can not be passed to the MediaFiles class. It's a challenge I'm sure I'll figure out, but for now, I'm stuck with some repeated code.

I'm also having trouble rendering JCheckBox's in my SongTable class. It seems that a default JTable cell will render a JCheckBox when a Boolean object is passed into a table row, but if you want to change how the cell's look with a DefaultTableCellRenderer class the JCheckBoxes don't seem to work right. Another challenge, this one though is not really necessary as you can still select rows...so that might go to the chopping block for now.

Lastly, the Java default icon is lame. I've been thinking about a new icon for the DSM, but I'm not an artist. I spent a few minutes messing around with an icon idea and I've drawn a few sketches but that's about it. Anyone want to design an icon for me???

I'm also getting comments on my youtube video. I appreciate those - it lets me know that people are getting use out of the program!

Brian

Saturday, October 17, 2009

JTable work today


Worked on the JTable that display track information. It's much prettier! I'm also cleaning up my code. It's all working still, but I think next week I'm going to go back to making it robust.

EDIT: I forgot to mention...I also turned on table sorting, so a user can sort the table by clicking on the table headers. A nice feature.
Brian

Thursday, October 15, 2009

Improved iTunes DSM Folder Watching

Big update today! I'm still working on the portion of the program that adds tracks from other directories to the iTunes library. You can see in the screenshot the new tabbed interface. It may not be my favorite way of doing things but its a simple way of it.

The new folder watch mechanism does what you think it would. iTunes DSM will now scan a directory for files/songs that are not in the iTunes library and add them to iTunes. It first creates a playlist, adds the song to the playlist, which are then added to iTunes.

The playlist must be created as that is the only way to programmatically add songs to iTunes, but once the process is completed you can delete the playlist.

I'm still working out the kinks and the code is really ugly but soon, I'll post a new version. Weekend is coming up! Hope everyone has a good one!
Brian

Saturday, October 10, 2009

iTunes DSM Watched folder ...esque

Today, I've been experimenting still on a watched folder implementation for the iTunes DSM. At my house, sharing the single music folder over the network, it can be a pain adding songs when one user has bought a new track and it doesn't show up on other's iTunes. The implementation so far seems to work, of course it's all command line based at the moment. Look forward to a usable gui implementation in the near future though.

I also seperated out some of the logic in the program. Searching for media files was once the task of the TracksTask class, but has since moved to it's own MediaFiles class. This has many obvious advantages, one thing that took a while to fully understand is that the MediaFiles class searches for files recursively as before, but the class creates an ArrayList of File arrays. So basically, there is one array with a bunch of arrays in it. The arrays inside the array list are essentially the "folders" that were found within the root music directory and those folders have the filtered files in them. Doing it this way allowed me to maintain the duplicate searching method the same as it operates on a "folder" of files as well.

Anyway, I registed for Google analytics the other day and I can see the traffic my blog is getting. It's kind of neat to watch the traffic increase and hopefully people are getting a use out of my program! Hope everyone has a good weekend.

Brian

Wednesday, October 7, 2009

What's in the future?

I haven't had much time to work on iTunes DSM this week. I've had exams and projects due, but every now and then I'll open up iTunes DSM and work on it. I've been cleaning up code, and adding documentation. I've also been exploring what I want to add next. I've got a couple of ideas.
  1. It would be handy to check the folder for files that have not been added to the iTunes library yet, so maybe some kind of folder watching mechanism. The folder watching mechanism will require some addition code but I do not think it will be difficult to implement.
  2. The options a user is able to choose from is increasing exponentially every time I add components so a better way to handle all the multitudes of operations is going to be needed.
  3. The GUI looks lame.
  4. The default JFileChooser object used on Mac operating systems also looks lame.
I have plenty of options to choose from and like most projects, it is usually new features that are added first before GUI updates are done. I'm almost trying to make this project my capstone project rather than my Instant Messaging application I was developing. It's been a while since I have worked on that and I'm simply not interested in it. The goal was to create something anyone could setup but it's just not viable with all the other instant messaging applications out there.

Saturday, October 3, 2009

First release available for download

Figured out how to host my program through my school's servers today. Here's how to use iTunes DSM.

  1. Be sure java is installed on your computer and you have JRE 1.6. Mac users you must set this property as it seems OSX uses 1.5 by default. Search in Finder for Java Preferences.
  2. Download and unzip the application from here http://students.uwf.edu/brg4/iTunes DSM .1.zip
  3. Run MusicApp.jar by double clicking it.
  4. Open your iTunes music directory.
  5. Click Find Duplicates
For a demo check out the YouTube video here http://www.youtube.com/watch?v=TQy-PXUEzjc

DISCLAIMER: It goes without saying, I am not responsible for any damages to your music library. In my informal testing the program appears to function correctly, but as always make a backup of your data before using iTunes Duplicate Song Manager.

Friday, October 2, 2009

iTunes DSM Video Uploaded

The demo video has been uploaded to YouTube. Here's the link...

http://www.youtube.com/watch?v=TQy-PXUEzjc

Monday, September 28, 2009

Demo Video Recorded for YouTube

I recorded a video of the iTunes DSM in action. I'm planning on posting it to YouTube to show how to use the program. In my testing, the program seems to be solid and ready for use.

I did use iTunes DSM on my own personal library finally and am pleased with the result. It's also convenient in that you can use the program on the other computers in the house and fix the orphaned files in the iTunes databases!

When I post the YouTube video I'll post it on here as well. I am still trying to figure out if I can host the program here on blogspot. I haven't really seen a way to do it yet.

Brian

Sunday, September 27, 2009

iTunes functionality in place.....for Windows

Using the previously mentioned components that I added, iTunes Duplicate Song Manager, can now locate orphaned files (those that have exclamation marks next to the song in iTunes) and remove those tracks. You can see the option added just above the clean music folder button.

The user can also just run the program and just clean the orphaned files, without having to scan for duplicates, so that's nice. The program also makes sure that the host operating system is Windows based, if not, the orphan file option is greyed out.

Finally, I've added a couple more links under the help menu.

I'm still wanting to add the orphaned files remove option to Mac but not really sure how to go about AppleScripts, so I'm sure that will be later. The program itself was having trouble running in Mac under java and I wasn't sure why at first. The SwingWorker package wasn't being found, turns out that Mac Leopard was using JRE 1.5 instead of the required 1.6, so if anyone else has problems using SwingWorker in OS X look for the Java Preferences application in Finder and edit the options in there.

In other news, my research paper is 4000 words, so that's nice too. Hope everyone had a good weekend.

Brian

Friday, September 25, 2009

iTunes functionality on the way

I know I haven't posted the project yet for download, and that's because I've been messing around with JACOB, Jacobgen, and the iTunes' COM. JACOB provides a way for the java virtual machine to interact with COM objects. This isn't really a java thing and so adding this functionality via JACBO brings java up to the likes of other development environments. You can find the JACOB project here...

http://sourceforge.net/projects/jacob-project/

it works for both windows 32 and 64 bit environments which is cool. Also on the sourceforge project page is jacobgen, which automatically creates .java files for the COM objects. So everything that the iTunes com offers, jacobgen will create java files with the corresponding methods to interact with it. Very Cool!!! I had some trouble getting it to work for a long time. There is an example batch file included in jacobgen that I had to edit.

Here is the batch file code I used to run jacobgen...


@echo off
cls

REM run this from the root directory of the Jacobgen project
REM it will spit out the interface classes for a dll you pass in as a parameter
REM sample command line while sitting in the JACOBGEN project directory
REM
REM The following command built a sample in the jacob directory I have
REM installed near my jacobgen project directory.
REM $ docs/run_jacobgen.bat -destdir:"..\jacob\samples" -listfile:"jacobgenlog.txt" -package:com.jacobgen.microsoft.msword "C:\Program Files\Microsoft Office\OFFICE11\MSWORD.OLB"
REM
REM
set JAVA_HOME=C:\Program Files\Java\jdk1.6.0_16
set JRE=%JAVA_HOME%\bin

set JACOBGEN_HOME=.
set CLASSPATH=%CLASSPATH%;%JAVA_HOME%\lib\dt.jar;%JACOBGEN_HOME%\jacobgen.jar;%JACOBGEN_HOME%\lib\viztool.jar
REM put the dll in the path where we can find it
set PATH=%PATH%;%JACOBGEN_HOME%\AMD64\

@echo on

echo %1
echo %2
echo %3
echo %4
echo %5




rem %JRE% -Xint com.jacob.jacobgen.Jacobgen %1 %2 %3 %4 %5

java -Xint com.jacob.jacobgen.Jacobgen %1 %2 %3 %4 %5

pause


REM run.bat -destdir:"c:\jacob\itunes" -listfile:"jacobgenlog.txt" -package:com.apple.itunes "C:\Program Files (x86)\itunes\itunes.exe"


To use the batch file. Open a command prompt and change directories until you get to the jacobgen directory. Place the batch file in the root jacobgen directory. Copy and paste the last line minus the letters REM into the command line and press enter. Jacobgen will place the created java files in the destdir. The main difference between my batch file and the example one is I use the java command.

Of course in the end, I didn't use the generated code. There is no documentation created with it obviously, and all kinds of errors were thrown in Netbeans. Instead, I downloaded this package http://dot-totally.co.uk/software/itunescon/ and in conjunction with JACOB, I can now control iTunes. I already have my program identifying the audio file in the iTunes library that correspond to the duplicate files found when searching for files with iteration counts appended to the end, so in conjunction with deleting the actual files, I should now be able to remove them from the iTunes library.

My next idea is since I share music throughout my home network, deleting files from the folder from one computer's library will not be reflected in other iTunes libraries until a user tries to play a song that is removed and therefore orphaned. In essence I'm going to try to add an option that will locate orphaned files, even if they are not apparent to the user.

Tuesday, September 22, 2009

iTunes 9 and an Algorithm Update

I've been using iTunes 9 since it came out and recently I've been trying to get it to create duplicates like the previous versions did. It seems at least that iTunes is much smarter about songs that are already in your library and basically seem to stop duplicates from happening; however, I did find that songs you place in the "Automatically add to iTunes" folder will create duplicates if the song is already present. I'm not sure if that is by design or not, but in the end iTunes seems to be better at preventing duplicates from occurring.

What about duplicates that were already present?
iTunes doesn't do anything for those, it still treats previous duplicates as seperate songs and so are added to the library. So for me, the 220 or so actual duplicate songs that were already in my library are still there.

I also made an obvious change to the search algorithm. The algorithm used to scan every file in all the folders of a parent directory adding each one to an ArrayList, then duplicates would be found by comparing each file to every single file in the list. This was a long process and required a lot of computer resources. Instead, the algorithm still scans each directory under a parent directory, but gets the list of files only in one folder compares those files for duplicates, and then moves on to the next folder and checks that one folder for duplicates. This has the advantage of finding duplicates as the folders are searched rather than reading every directory first and then comparing every single file to each other. This works because as iTunes keeps folders organized, there is no way that duplicate song will end up anywhere but in the same folder as the original.

I'm still planning on releasing the program here real soon. I have decided that it will be free. I have placed a button for donations on the program and on this blog, if you are feeling generous, I could use a cup of coffee every now and then! I will also post a video on youtube on how to use the program. I'm hoping to do this this week.

Brian

Sunday, September 20, 2009

Bug fixing



I've been updating the program more this weekend. There were a few sizable bugs I found. One of the major ones occurred when the user didn't select every song to be cleaned, rather a few of the songs. Instead of deleting the selected songs, the program would delete the number of selected songs starting from the top of the list. For example, if I picked three songs sprinkled throughout the list, the program would delete three songs starting from the top. In the end it was a minor fix and that portion is working correctly.

I've come across the Swing Worker class in Java. Swing Worker provides a way to utilize threads in the background so that the main gui thread is not bogged down with operations not directly related to the gui. In essence, the program appears to run much faster and smoother. Also a part of the Swing Worker class, the user can now cancel functions and more easily restart searches.

I have also tweaked the algorithm that finds duplicate songs. I still have artist, title, and album matching turned off as it greatly increases the computations. The program does still search filenames that end in iterations though.

Warning messages now appear if selections haven't been made and there are a couple title menu options as well namely, exit and about.

I have named this version currently version .1. I'm not really sure to version conventions, though I know v1, v2, etc are full released versions and everything in between usually refer to minor releases or bug fixes. Basically, I picked an arbitrary number and went from there.

I included a few screenshots, notice the cancel button and the title menus.

What's Next?
I'm looking into logging, to keep a record of what happens. Also a major convience to the program would be to not only remove the file itself but also remove the song from the itunes library, but that's a ways off, as communicating through com's in windows isn't really java's thing. As of now, to the have the library updated, the user has to delete the songs from the library(not from the hard drive) and then reimport the songs, so that iTunes knows exactly what is in the folders. That's all to come and until then...toodles.

Brian

Wednesday, September 16, 2009

GUI Update Again



Had some time to work on the iTunes duplicate song manager program today "betwixt" the paper and other homework. It is now much easier to actually see that the program is doing something while its processing in the background. The screenshot above shows the added progress bar and the current file or directory the program is scanning. Still not quite ready for an alpha release or anything but it's getting there.
Brian