A functional introduction to CEDET – EDE

CEDET is a collection of elisp packages for Emacs designed to turn it into more of an IDE. While it does have the ability to better bridge the gap between editor and IDE, it can be an enormous pain in the ass to figure out; it is “documented” in a rather haphazard way, and resembles an API more than a UI, and as anyone who had to work with uncommented code knows, this is not exactly a recipe for fun.

There’s an article called “A Gentle Introduction To CEDET”, which is pretty good and helped me out in understanding the cacophony of features that is CEDET, but it certainly wasn’t enough. I also spent time reading the manuals for the various sections and had to play with the features before I finally began to comprehend how to get CEDET to do the things I installed it for, and the main take away from this experience was that CEDET has horrible usability, but it’s the best tool we have for enriching Emacs with IDE features, so someone better write an article that addresses CEDET from a functional point of view (i.e. I want to do X) rather than the “understanding the internals” point of view that too often GNU projects take. Today, I’m going to cover EDE, CEDET’s project management module.

Project management

Setting up project management is the very first thing anyone should do to use CEDET, because many of CEDET’s tools use the project definitions to get necessary information. Project management is handled by the EDE component, and supports the following features:

EDE features

  • Manage makefiles: add/remove targets, add/remove files to targets.
  • Project configurations (i.e. debug build, release build)
  • Create distribution file
  • Initiate building, debugging, or running of the project.
  • Viewing and updating project metadata (this includes anything from giving the linker special options to choosing a compiler to updating the project’s version number). This feature is meant for getting down and dirty with EDE, but there’s a GUI for this, so you don’t have to mess around with elisp. Note: this section is described near the end of the post.
  • Searching for files in a project
  • Viewing project tree organized by targets
  • Managing the project’s web site. You can define a “web page file”, a host address, an FTP server address, and a mailing list address for your project. EDE includes the feature to update the project’s website and upload the distribution.

To turn EDE on, place the following code into your .emacs file:

(global-ede-mode t)

When EDE mode is on, and you visit a project, you will see a “Project” option appear in the menubar. Many of the commands I’m about to describe can be accessed via that dropdown menu.

The EDE world view

There is one important thing to understand about EDE, and that is the way in which it looks at your projects.

EDEprojectmodel

In EDE’s view, a “project” is files within a folder + their makefile. If you have multiple folders in your codebase, then each one of those folders needs to be defined as a “project” as well, and these “projects” are considered to be subprojects of the top level project.

It’s worth mentioning that EDE is aware of things that happen down the folder hierarchy. You can be at the top folder, initiate a file search, and it will search the entire hierarchy. When you add a subproject, the makefile for the top project is automatically updated to build the subproject as well.

Supported EDE use cases

The question you need to ask yourself when you decide to define a project is which one of the following scenarios fits you:

  1. I’m starting my project from scratch, and want EDE to manage it for me.
    This mode of operations is the most feature complete because as you write your code you set everything up the way EDE wants it to be. This option lets you create either Makefile projects or Automake projects.
  2. I have existing source code that EDE knows how to work with.
    This mode of operation is good for Automake projects, and 2 specially supported projects: Emacs and the Linux kernel. In these situations you need to just go to the source code and EDE will identify everything automatically. This mode of operation is less feature complete, in part because it assumes that since you have existing makefiles, you probably want to keep managing them yourself.
  3. I have existing source code that EDE doesn’t know how to work with.
    In this mode of operations none of the project management features will work, but you can still use EDE to define your code as a project. The reason for doing this is so that some limited amount of metadata will be available for other tools in CEDET. This mode of operation will require you to write elisp code, but you can have a minimal setup with very few lines of elisp. I’ll explain how to do this easily further in the post (there’s a template). Since C/C++ projects with existing Makefiles are common, EDE has a special “project mode” for them.

Note: EDE now supports Android and Arduino projects too. These features appear to be relatively new, and I will not cover them in this tutorial.

With that in mind, let’s go over the 3 use cases…

EDE with new projects

The EDE documentation has a quick start section that is worth looking at for a short example of how to work with EDE for new projects. The general gist is as follows:

Suppose we have a folder hierarchy as depicted in the picture above: a folder with c files and a header, and two subfolders, also with c files and a header in each (currently there are no makefiles or Project.ede files). You need to do the following:

  1. Open a file in the top level folder, and type M-x ede-new. This will ask you for a project name and type: make or automake. This creates a Project.ede file in your directory.
  2. Type M-x ede-new-target or C-c . t, and it will ask you for a name, target type (see below), and if you want to add the current file to that target.
  3. Repeat step 2 as needed to create all the targets you want.
  4. Open a file in the current directory, and type M-x ede-add-file or C-c . a and it will ask you which target you want to add this file to.
  5. Repeat step 4 as needed to add all the files to the necessary targets.
  6. Repeat steps 1-5 for each subfolder, and then their subfolders, and so on.

Supported targets for ede-new-target:

The command ede-new-target requires you to specify what kind of target it is. The following is a list of possible targets and what they are meant for (thanks to Alex Ott from stackoverflow.com for the list).

  • archive – object code archive/static library
  • auxiliary – handles auxiliary files in project – README, *.txt, etc. When I played around with this type of project the header files I added to a “program” target ended up here as well. It looks like this is for files that don’t get compiled.
  • emacs lisp – compiles Emacs Lisp code
  • emacs lisp autoloads – target to create file with Elisp autoloads
  • info – handles texinfo files
  • miscellaneous – target that allows user to compile other parts of project, using user-provided Makefile (one that is not generated by EDE)
  • program – builds executable program
  • scheme – target to handle Scheme (guile) source code
  • semantic grammar – target to correctly build Semantic’s grammar files, both for Wisent & Bovine. This target is defined in Semantic, not in EDE
  • sharedobject – target to build shared libraries

Supported features:

The following is the complete list of features that EDE supports. When working in this manner, they should all work or be supported:

Create target: Creates a target for your makefile, and possibly adds the current file to it.
Invocation: M-x ede-new-target or C-c . t
Remove target: Removes a target from a makefile.
Invocation: M-x ede-delete-target
Add file to target: adds the current file to a target of your choice.
Invocation: M-x ede-add-file or C-c . a
Remove file from target: removes the current file from a target of your choice.
Invocation: M-x ede-remove-file or C-c . d
Building: build the current project or target. You can access more target building options from the menus.
Invocation: C-c . c for current target, C-c . C for entire project.
Debug target: Run a debug session on current target.
Invocation: M-x ede-debug-target or C-c . D
Run target: Run the current target
Invocation: M-x ede-run-target or C-c . R
Create distribution file: Creates a tar.gz file with your code in it.
Invocation: M-x ede-make-dist
Locate files: search for a file in your project.
Invocation: M-x ede-find file or C-c . f
Speedbar: Open a project hierarchy navigation side buffer that shows your files grouped by targets.
Invocation: M-x ede-speedbar or C-c . s
Make distribution: This command will create a tar.gz archive of your project.
Invocation: M-x ede-make-dist
Upload distribution: This command will upload the distribution file to the FTP server you define. Using this feature will require you to define the FTP server’s address via the customization feature, which is covered later.
Invocation: M-x ede-upload-distribution

A personal note:

Having played around with EDE for a bit my impression of it is that it’s cute, but not overly useful. While the initial creation of targets is kinda nice, the moment you decide that you want to do something more you’ll quickly find EDE getting in your way, because it’s nothing more than a layer of (bad) abstraction over the Make system. I’m saying “bad” because an abstraction should mean that you create a new, simpler “language” that the user can use without knowing what is underneath it, but in the case of EDE it doesn’t hide any of Make’s complexities from the user, but rather gives you a different way to specify the things that you want to do, and probably already know how to do by hand anyway (not to mention that unless you know how makefiles work, you’re not going to be able to make any sense out of the customization buffers of your targets).

For example, suppose that you need to compile your code using special flags (-Wall) and link a bunch of libraries (-lm). In EDE this means opening the target’s customization buffer and searching for the option to specify libraries you want to link, and flags you want to pass to the compiler. If you already know what these options mean, you already know how to type them in the Makefile and where, and it would be much more natural to just do that instead of figuring out what’s the “EDE way” of doing the same thing.
What makes this even worse is that the way you specify some options is by overwriting the default values of Makefile variables that EDE generates, and how do you know what these variables are and how they are used? By looking at the Makefile of course! Yes, some of them are pretty standard and once you learn them you don’t need to check the Makefile anymore, but the point is that if you need to reference the internal structure of the Makefile then what kind of an abstraction is this? How is this helping us not mess around with Makefiles (which is presumably the reason we want EDE to manage them for us)??
And if this wasn’t enough, there’s also the myriad of Make features that EDE doesn’t support, which is a headache in its own right if you want to do anything remotely non-trivial. This includes basic stuff like initializing a variable using ?= for example. The EDE workaround for this is to let you define files to “include” into the Makefile, which basically means “write all of that stuff in your own Makefile, and tell EDE to include it in the Makefile that it generates”. Again, if the way to do these things is by hand, it’s going to be much more natural to just write it in the main Makefile, and not write a second Makefile and include it in the first.

In conclusion, this method of working is sort of kinda OK if you want to do some simple stuff, but I can’t imagine any serious project with a degree of non-trivial complexity using it, and quite frankly you’re better off doing the simple stuff by hand as well, because EDE is just going to get in your way, and its extra features like making an archive of your code and uploading it to an FTP server are far from being “killer features” that will make the hassle worth it.

EDE with existing code that EDE doesn’t recognize

For any not-officially-supported scenarios EDE has the notion of a “simple project”, which is basically a do it yourself option. ede-simple-project is an EIEIO object that is designed to be subclassed, and you are free to implement all the necessary elisp functions by hand to support whatever situation you have in your project. If you know how to do that, you don’t really need mu help.

Simple projects are defined by writing elisp code in your .emacs file, and are pretty much featureless. None of the major commands like building and creating targets will work, but the more trivial ones like “Edit Project Homepage” or “Browse Project URL” will, as long as you manually set the appropriate variables in your elisp code (since there’s no Project.ede where settings can be persistent, it is not possible to set this via the customization features of EDE). However, like I said at the start of this post, you will still want to use a simple project even if you don’t plan to extend it, just so that other parts of CEDET will have access to the limited metadata it does have. Otherwise, you may have to configure all the other CEDET tools by hand.

I will however show you how to setup a simple C/C++ project, which doesen’t take much knowledge to do. Its EIEIO object ede-cpp-root-project is special because it’s aware of some typical C/C++ idiosyncrasies like the abundance of .h files. A basic configuration can be created like this:

(ede-cpp-root-project "project_name" :file "/path/to/some/top/level/file"
         :include-path '( "/include" "include/configs" )
         :system-include-path '( "/usr/include/c++/3.2.2/" )
         :spp-table '( ("MOOSE" . "")
                       ("CONST" . "const") ) )

Hopefully most of the stuff you see in this code sample is self explanatory even if you don’t know elisp. Essentially what this code does is give the project a name, tell CEDET where the the project root is by pointing it to a file in that root directory, tells it where the include files are (an entry starting with a “/” is relative to the project root, while an entry that doesn’t start with “/” is interpreted as relative to the current buffer), tells it where the system include files are, and then there’s that spp-table variable.

spp-table is where you can list a bunch of C preprocessor defines that are critical for the various #ifdefs in your code. CEDET can theoretically find them while parsing the code in your project, but placing them here explicitly makes sure that they are indeed found when needed.

You can see a full documentation of ede-cpp-root-project in the EDE manual under the Project/ede-cpp-root-project entry. Examining the inheritance tree can give you the full list of elisp “slots” (variables) you can configure in the above template. The full list is this (items you probably not need to set are grayed out):

  • name: String
    Default Value: “Untitled”
    Used for distribution files. No reason to set this since Make Distribution is not supported in simple projects.
  • version: String
    Default Value: “1.0”
    Used for distribution files. No reason to set this since Make Distribution is not supported in simple projects.
  • directory: String
    Directory this project is associated with. Not sure what its use is considering that this can be inferred with the help of the file variable.
  • file: String
    A name of a file in a top level directory of the project.
  • targets: List
    List of top level targets in this project. This is used by EDE when it generates Makefiles, and I’m not sure of whether you have a reason to set this or not in a “simple project”, although I suspect that it might help other tools find some stuff better if you do (I myself do not bother to set it).
  • tool-cache: List
    List of tool cache configurations in this project.  This allows any tool to create, manage, and persist project-specific settings. If I’m not mistaken this section is for plugging in custom tools (notice that the “tools” section of customize-project is always empt).
  • web-site-url: String
    URL to this projects web site.  This is a URL to be sent to a web site for documentation.
  • web-site-directory: String
    A directory where web pages can be found by Emacs.  For remote locations use a path compatible with ange-ftp or EFS. You can also use TRAMP for use with rcp & scp.
  • web-site-file: String
    A file which contains the home page for this project.  This file can be relative to slot `web-site-directory’.  Same guidelines as above.
  • ftp-site: String
    FTP site where this project’s distribution can be found. This FTP site should be in Emacs form, as needed by `ange-ftp’, but can also be of a form used by TRAMP for use with scp, or rcp.
  • ftp-upload-site: String
    FTP Site to upload new distributions to.  Same guidelines as above. If this slot is `nil’, then use `ftp-site’ instead.
  • configurations: List
    Default Value: (“debug” “release”)
    List of available configuration types.  Individual target/project types can form associations between a configuration, and target specific elements such as build variables. Not useful to you unless you plan to write your own elisp functions that use this.
  • configuration-default: String
    Default Value: “debug”
    The default configuration. Not useful to you unless you plan to write your own elisp functions that use this.
  • local-variables
    Default Value: `nil’
    This variable is for setting Emacs variables that you want be set only for this specific project.
  • include-path: List
    Default Value: `(quote (“/include” “../include/”))’
    The default locate function expands filenames within a project.  If a header file (.h, .hh, etc) name is expanded, and the `:locate-fcn’ slot is `nil’, then the include path is checked first, and other directories are ignored.  For very large projects, this optimization can save a lot of time.
    Directory names in the path can be relative to the current buffer’s `default-directory’ (not starting with a /). Directories that are relative to the project’s root should start with a /, such as  “/include”, meaning the directory `include’ off the project root directory.
  • system-include-path: List
    Default Value: `nil’
    The system include path for files in this project.  C files initialized in an ede-cpp-root-project have their semantic system include path set to this value.  If this is `nil’, then the semantic path is not modified.
  • spp-table: List
    Default Value: `nil’
    C Preprocessor macros for your files.  Preprocessor symbols will be used while parsing your files.  These macros might be passed in through the command line compiler, or are critical symbols derived from header files.  Providing header files macro values through this slot improves accuracy and performance.  Use `:spp-files’ to use these files directly. I myself do not bother setting this.
  • spp-files: List
    Default Value: `nil’
    C header file with Preprocessor macros for your files.  The PreProcessor symbols appearing in these files will be used while parsing files in this project.  See `semantic-lex-c-preprocessor-symbol-map’ for more on how this works.
  • header-match-regexp: String
    Default Value: “\\.\\(h\\(h\\|xx\\|pp\\|\\+\\+\\)?\\|H\\)$\\|\\<\\w+$”
    Regexp used to identify C/C++ header files. If you need to set this then wat.
  • locate-fcn: function (or null)
    The locate function can be used in place of “ede-expand-filename” so you can quickly customize your custom target to use specialized local routines instead of the EDE routines.  The function symbol must take two arguments:   NAME – The name of the file to find. DIR – The directory root for this cpp-root project.
    It should return the fully qualified file name passed in from NAME.  If that file does not exist, it should return nil.

EDE with existing code that EDE recognizes

As mentioned before, there are a few use cases for this option: Automake projects, Linux project, and the Emacs project. It is worth noting that in EDE’s eyes, existing Automake project and Linux/Emacs projects are actually two different things. Linux/Emacs projects are essentially “simple projects” that are preconfigured and prepopulated with symbols.
I don’t have much to say about this option. It’s going to be somewhere between “simple project” and a fully featured project type.

Advanced: Low level customization of your project

Most of EDE’s project metadata isn’t accessible from the high level commands described above, so in order to customize it you either have to edit the project definition by hand, or preferably use the customization feature. This feature worksin two possible situations:

  1. Your EDE setup uses a Project.ede file (use case number 1).
    If your setup uses a Project.ede file, the customization feature will take you to a special buffer similar to Emacs’ “customize” buffer.
  2. Your EDE setup does not use a Project.ede file (use cases 2 and 3).
    In this situation EDE is either using Makefiles or some other file to learn about the project’s setup, and the customization feature will take you to these files themselves. For example, if you are in the Linux project, and type C-c . e, it will take you to scripts/ver_linux.

The customization buffers can be accessed via the Project menu, and look something like this:

customizationbuffer

As you can see, the different options are documented. Some of the documentation in there is helpful, other not so much. Since this post is already long enough, I’ll focus on describing what I personally feel is the non-obvious stuff.

  • Default:
    • For Automatic-Dependencies, Include-File, and Inference-Rules you can view this post.
  • Settings:
    • Variables: literally insert variables into the makefile.
    • Configuration-Variables: insert variables that will only be used when compiling with the appropriate project “configuration”.
    • Project Local Variables: set Emacs settings that will be relevant only for this project.
  • Targets:
    • Auxiliary Files under “Local Targets”: when I added header files to my project, they ended up here, which is not what I would’ve expected given the description in the documentation.
    • Linker and Compile options: I didn’t set any, and it appears that this causes EDE to default to GNU tools like GCC.
    • Additional Rules is for inserting hand written rules.
    • LdLibs: despite what the documentation says, this is perfectly supported in Make projects as well.

Confused yet? I don’t blame you.

Conclusion

If you want my opinion: don’t bother working with EDE directly. It’s not really worth it. Just write your own Makefiles, and use EDE’s ede-simple-project or ede-cpp-root-project to define where your includes are, and if you want to- what you web site URL and FTP address is for all those cutesy features. If you don’t want to pollute your .emacs file with project specific definitions, you can do the following:

Place this code into you .emacs file:

;; Create a Project.ede equivalent for ede-simple-project
;; by telling Emacs to load Project.el files
(defun check-for-Project-el ()
  (if (file-exists-p "Project.el")
    (load-file "Project.el")
  )
)
(add-hook 'find-file-hook 'check-for-Project-el)

This function makes Emacs check for the existance of a Project.el file whenever it opens some file, and loads its elisp code. Now, place a Project.el in your project folder, and write all the configuration there.

So that’s it for now. Next time I’m going to cover Semantic.

Posted in Open Source, Software | Tagged , , | 3 Comments

Software engineering in the Technion – semester 5

This is the fifth post in my series about studying software engineering in the Technion. If this is your first time reading a post in this series I recommend that you read the introduction post first (go ahead, it’s rather short). If not, I’ll give you a short reminder of where we are:

The fifth semester is in the “second tier” of the degree, which is where we concentrate on the essential computer subjects of the degree.

All the courses in this post are mandatory (including Biology 1, in the sense that we are required to study 8 academic points of science courses. The choice of Biology 1 as my final scientific course however was mine). 

Table of contents:
  • Parallel and Distributed Programming 
  • Theory of Compilation
  • Computability Theory
  • Introduction To Software Verification
  • Numerical Analysis 1
  • Methods In Software Engineering
  • Biology 1

The fifth semester

frame

Parallel and Distributed Programming
3 academic points. Computer science faculty course.
Prerequisites: Operating Systems.

What is it about: The challenges of programming systems with multiple processors.

About the course:  Parallel programming (when there are multiple processors working on the same task simultaneously) is the new frontier, because the method of improving performance by literally making hardware faster is not practical anymore. Today, if we want to continue getting better performance out of computers, we have to do it by utilizing multiple processors, and that brings with it a bunch of tricky challenges: we want to distribute the workload evenly to get the maximum use out of our multiple processors, but we can’t always know in advance how to distribute the work in such a way. We can try to distribute the work as it happens, but now we have a new problem: managing the whole thing, which can lead to a lot of costly communication between the threads. We also have to deal with memory issues: in some systems memory is divided between the processors; accessing your own memory is fast, but accessing the memory of other processors is slow. How do we distribute the workload in such a way so as to minimize how often we need to get data from someone else? And then again, in some systems you can’t access somebody else’s memory, so in order for them to share information we need to set up communication between the processors, and this is a problem in itself (how to do this safely, how to do this sparingly).

parallelProblem
A simple division of work between multiple processors.
 

This course teaches how to deal with these things, and covers, along with the theory, a bunch of libraries and APIs that help us handle the challenge of parallel programming. I was pleasantly surprised to learn that there are some very useful libraries for concurrent programming. Up to this point I was used to doing everything manually: create threads, use shared data, synchronize it manually, etc etc. It can be very tricky work, and the libraries they showed us in this course did help with some of the challenges.

Of course, the incredibly low difficulty of the assignments might’ve contributed to my favorable impression of these libraries. I think that overall the lecturers in the computer science faculty are perfectly fine (some even good), but something went really wrong with this particular course. The lecturer was apparently crappy (I wouldn’t know because I didn’t go to most of the lectures, but I heard more complaints than usual), the assignments were ridiculously easy to the point where they didn’t contribute to our understanding of the material, it took forever for the staff to check the homework (assignment 2 took 10 weeks!!), the TA in charge of most of the assignments was extremely unhelpful and you could barely get an answer out of him, there was confusion about what material is going to be in the exam, and as if that wasn’t enough they gave us an unsolvable question in the exam. There was a lot of outcry about this, and a representative of the student union put a lot of pressure on the lecturers to remedy this, so in the end we got a very forgiving exam check (one multiple choice answer had 15 possible answer combinations that got full mark!), and we got to enjoy the ever so rare square root factor (a “factor” is a correction to the grade that is introduced to bring the course GPA to a certain level. Usually a factor is something like +5 points, or +10%. A square root factor is grade = 10 * sqrt(grade). I don’t know the mathematical rationale behind it, but it’s one of the strongest factors you can get). But in the end of the day, this was just treating the symptoms. On the previous semester 60% failed the exam. They got a factor of final_grade=exam_grade*100/86+5, and as you can see, the crappiness repeated itself on my semester.

What is it good for: Parallel programming is extremely important and can be found in many programming problems, from networking to GUI to operating systems, to many other things (this course focuses a bit more on the dealing-with-hardware-related-challenges than the purely software related problems though).

compilationTheory of Compilation
3 academic points. Computer science faculty course.
Prerequisites: Automata and Formal Languages.


What is it about: How to write compilers. What’s a compiler you ask? I’ll tell you. This is how programs are written nowadays: programmers write the program in some programming language like C/C++/C#/Java/Pascal/whatever, which is easy for humans to read and understand but means nothing to the computer, and then a program called a “compiler” is activated and translates it into computer language, which is hard for us humans to understand (thus the reason why nobody uses it to write programs) but the computer understands perfectly.

About the course: I’m not really into compilers so I couldn’t get too excited about this course, but objectively I can tell that it has nice things about it. For example, you get to write a really simple compiler in one of the homework assignments, which is cool. Another interesting part was the general structure of compilers; turns out that compilers are really modular things, and there are even various tools that can generate some parts of compilers automatically: you give them a language definition and they generate the appropriate parser. This is possible thanks to the theory you studied in the course Automata And Formal Languages, which shows that this seemingly archaic theory actually has really concrete applications.

compilerThe stages of compilation.
 

But most of all I enjoyed the part where the lectures ended a full month ahead of time. Turns out that this course is relatively short on material, which was a refreshing change. It’s not often that you get some kind of reprise in the Technion.

What is it good for: Well, if you want to work on compilers, this is definitely the course to take. But it is also a pretty good demonstration of how to do parsers and the various tools that are available for the tasks. Like our TA said in one of the tutorials: sometimes you may get a file with raw data, and you will have to do a lot of work to make sense of it. The kinds of tools you learn in order to write compilers are also very useful for automating these kinds of tedious tasks (if you know how to use them).

computabilityComputability Theory
3 academic points. Computer science faculty course.
Prerequisites: Algorithms 1.


 
What is it about: This course covers two subjects: computability theory and complexity theory. Computability theory asks the question: which problems can be solved using a computer? Complexity theory asks: given a problem that can be solved using a computer, how much resources (time and space) do we need to solve it?

About the course: This is yet another course that is very logic-like, and it also happens to be my favorite logic-like course, because the theory here is really clever and elegant, with some pretty cool conclusions. Just the way it describes computations is quite remarkable:

Imagine that you have an ABC that consists of only two letters- 1 and 0, and you have a machine with a tape on which lots of 1s and 0s are written. This machine can read the tape and selectively change the “letters” based on some internal mechanism. (This machine is called a “Turing Machine” after Alan Turing who invented it. )

Sounds too simple to do anything useful? Well, apparently this deceptively simple mechanism is expressive enough to describe all computations. Just think about this for a moment. This seemingly dumb, limited machine can describe state of the art artificial intelligence. That’s quite amazing to me.

turingMachine
An artist’s representation of a Turing Machine.
 

In the first half of the course you learn that there are problems that can’t be solved (the halting problem), which has all sorts of interesting repercussions like the fact that it’s impossible to write a program that catches all bugs in another program. You learn that some problems are intrinsically harder than others, and that if you can figure out how to solve them you will be able to solve a whole array of other problems, which is an idea that continues in the second half of the course as you learn about complexity. You even get to see a mathematical proof that it is possible to write computer viruses (it’s possible to write a program that copies itself), or that operating systems are possible (you can write a program that runs other programs).

In the second half you learn that some problems require more resources in terms of time and space than others, and that there is a hierarchy of difficulty in here as well. You also learn the meaning of that iconic question in computer science: does P=NP, which in other (very simplified) words basically asks “given a problem, is finding the solution as difficult as validating the correctness of a given solution?”. If the answer is yes, then the whole field of cryptology will collapse (or at least a big part of it) because many methods of encryption rely on the assumption that the answer to this question is “no”. For what it’s worth, most experts believe the answer is indeed “no”, in part because it flies against centuries of experience in the field of problem solving. Scott Aaronson of MIT put it rather eloquently when he said:

“If P = NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in ‘creative leaps,’ no fundamental gap between solving a problem and recognizing the solution once it’s found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss…”

Scott Aaronson, MIT

This course had a few interesting idiosyncrasies. First one was that the exam had two parts, and you can attend which ever part you want as long as you do both parts by the end of the second terms. That’s a very welcomed change because it definitely reduces the amount of work you need to invest into the overall exam (since you can split it). A less welcome change was the fact that you didn’t get any grade for the homework. Instead, you could come to a “workshop” (AKA the TA’s office hours) and ask questions about your solution. While the personal attention was apparently much appreciated by the students, I really feel like this no grades thing was terrible, because no matter how much self discipline you have, it’s just too easy to postpone the homework indefinitely when you have pressure from other assignments for which you do get a grade. I consider myself to be pretty self disciplined (in fact, I often feel disapproving of how much less disciplined others seem to be), and yet even I ended up postponing the assignments until- well, until the exam times. The result was an almost perfect score on the first half, and a barely passing grade on the second half.

a_bunch_of_rocks

What is it good for: For a software engineer- not much. This is really a course for computer scientists. But one practical thing you will get from it is the ability to identify NP-Complete problems, which will help you to avoid trying to solve them (though it haven’t been conclusively proven they can’t be solved efficiently- that’s the P=NP question- no one managed to solve them efficiently yet, so you should probably give up and come up with an approximation solution).

whatIntroduction To Software Verification
3 academic points. Computer science faculty course.
Prerequisites: Logic and Set Theory For Computer Science.


What is it about: The theory behind methods for automatically verifying the correctness of programs.

About the course: This course is pure crap. Well OK, it’s not pure crap, but it’s pure crap that it’s in the mandatory course list. The stuff they teach you is out there, and very occasionally it’s even used in real projects, but most people will not meet it in their careers, and as such this really should’ve been an elective course.

The course is very logic like, but despite my general liking of logic I did not enjoy this one at all. It had some very annoying homework (akin to washing floors with a tooth brush), strange syntax and algorithms, and for some reason they decided not to have lecture slides like every normal course, and instead taught by writing on the board like it’s the middle ages. Copy stuff by hand?! Show up for lectures?! Fuck that shit.

The general gist of the course is this: it’s impossible to verify correctness in all cases, so we have a bunch of different methods for different situations. The first half of the course focuses on proving correctness (Floyd and Hoare methods), and the other half focuses on proving the model (temporal logic, CTL, BDD, and some other crap).

What is it good for: Only for working with automatic verification.

whatNumerical Analysis 1
3 academic points. Computer science faculty course.
Prerequisites: Introduction To Computer Science M, Algebra A, Calculus 2T.


What is it about: Something to do with doing advanced math with the aid of computers.

About the course: I didn’t take this course yet. Come back in 6 months.

What is it good for: —

softwarearchitectureMethods In Software Engineering
3 academic points. Computer science faculty course.
Prerequisites: Data Structures 1, Object Oriented Programming.


What is it about
:  The development cycle: its various stages (requirements definition and analysis, design, testing, and development), how to do them right, and how they are connected.

About the course: This is a pretty important course to take, because it’s an introduction to how software projects are managed in real life, and what kinds of challenges a software project faces (basically, it’s taught from the project management point of view, rather than the programming-task-specific view most courses take). It discusses the software life cycle, from design to development to testing to support, gives an introduction to UML (a graphical language for design. This takes the greater part of the course), covers various project management methodologies (extreme programming, scrum, waterfall model, rapid prototyping, etc), and discusses testing.

diagram02
Scrum is one example of agile development.
 

The homework assignments are used to simulate all the development stages that are taught in class: students are divided into groups of 5, and are given a “client story”- a description of a problem or a product that some fictional client needs solving/doing (in our cases it was something called  “GPSWiki: Motorists’ Joint Road Knowledge Base”, which was supposed to allow motorists to track their rides and submit them online), and the students are supposed to do the requirements and “use case” analysis, software architecture and deployment model, come up with a test plan and a work plan, and implement a prototype.

275px-Windows_2000_architecture.svg
The Windows 2000 architecture.

It was pretty cool in theory, but my experience of the process was soured by two members going AWOL on us, which left us unable to implement all the requirements on time. We the remaining three ended up sending an email to the TA in charge and got our grade “fixed” by a factor of 1.5, so I still got a descent grade for it, but it was after a lot of annoyance and resentment on our part. It was especially annoying to me because I was the project leader, and I was the one who had to deal with these two. On the following semester I- like most of the students of this course- took the first phase of the Yearly Project In Software Engineering, and the same thing happened in one of the groups that got one of the deserters. That group got broken up in the middle of the semester and their members were scattered between the other groups. The guy who arrived to our group told us that the deserter was given a task that could be done in 20 minutes, delayed it until the very end, and then failed to do it.

The moral of this story is: choose your partners well.

1415714204_cbd03aba23
Of course, real projects don’t always go according to plan either.


What is it good for
: software engineering in general.

biology1Biology 1
3 academic points. Biology faculty course.
Prerequisites: None.


What is it about
: This is the entry level course in biology. It focuses greatly on the anatomy of the cell.

About the course: I took this course as my “third scientific course” per the requirement to study 3 scientific courses, and it was definitely a refreshing change of pace. It involved a great deal of memorization, which is rare in my degree, and seemed to be from a completely different world to me. I literally felt like different parts of my brain were at work here.

It was pretty interesting; there was a lot of chemistry involved, which was surprising to me, but that too was kinda cool. I guess the thing that caught my interest the most was how incredibly complex cells are. There’s a whole array of various chemical mechanisms and organs (yes, cells have internal organs) that allow the cells to do very complex things: metabolize materials, expel waste, be selective in what they let in, divide (which is a much more amazing feat once you know that cells have internal organs), and more. The chemical parts had their own surprises as well; turns out the various materials have a very Lego like quality in the way they mix and interact, and some of the products of those interactions can take on functional properties beyond just being a mass of stuff. That’s how the original cells first originated.

 

eukaryotic cell
An Eukaryotic cell.


This course is notable for being a web oriented course. All the lectures are recorded and provided on the course site, so you don’t have to go to lectures at all and are in fact encouraged to study using the recorded lectures. Each lecture has a forum, and a selection of questions. There were also more formal homework assignments but you didn’t get a grade for them, and instead the solutions were published later on for you to test yourself. Apparently this whole thing is part of a pilot, testing the idea of making this course entirely web based (replacing formal lectures). Not sure what the decision will be, but it sure felt quite possible to me.

What is it good for: Well, not much for a software engineer, though it’s important for someone who wants to study bioinformatics (a combination of biology and computer science).


So that’s the end of the fifth post. Hope you liked it. To find the rest of the series:

Previous entry Index Next entry

Posted in Academia | Tagged | Leave a comment

Software engineering in the Technion – series introduction

So, you decided to study software engineering in the Technion.

PipBoy_T_Shirt_Design_by_TheCoconutG[1]

You will get to be one of Israel’s best and brightest.
Technion students get laid more often, earn tons of money, and kick ass at rocking being awesome. …and while the computer science faculty was ranked “only” 15th in the world by the Shanghai university ARWU, at least none of our graduates ever caused a resonance cascade scenario that brought down an age of slavery by otherworldly aliens on the entire planet.

But surely you have some questions lingering in your mind about what you will study and what the experience is going to be like, even if you already decided to study it based on some prior knowledge/experience. Alternatively, you may be a layman considering going into this field that you know nothing about, or you may be just curious about what software engineering is all about. This is where this series of posts comes in. In it I’m going to describe the curriculum of a B.Sc. in software engineering as it is studied in the Technion, and is going to be described in the same order as is suggested by the institute (you can do the courses in any order that is convenient to you provided that you always have the necessary prerequisites, and many people, including myself, deviate from the suggested order in practice). Where personal choice is given to the student (mostly in the later stages of the degree) I will describe the courses that I personally chose to do.

sw_thumb11

The B.Sc. degree takes 4 years to complete, and is divided into three tiers; the first tier consists of the first three semesters and is all about teaching you the fundamental mathematical theories and basics of computer science (hardware and software). Frankly, this part of the degree sometimes feels like you are doing a math degree and not a software engineering one, especially in the first year. These non-computery subjects tend to either outnumber or equally match the computery ones, and while there’s a reason for that it can be a bit bumming for some. The first year is also when you are considered to be in the general 4 year computer science program. You get to switch to the software engineering program once you accumulated the equivalent number of academic points of 2 semesters.

Then comes the second tier, which consists of semesters 4 and 5. It is then that you are taught the essentials of your profession (of course, the first tier has some computer stuff too, but now it’s all about computer stuff). The third tier is where you get to choose what to specialize in. You have about 20 academic points to spend on whatever tickles your fancy, and ideally they should all be spent on the 5 “core” courses from which you are required to choose only 2, but you can of course branch out if you please.

The official requirements are as follows:

  • 160 academic points, of which:
    • 118.5 are mandatory courses academic points.
    • 31.5 are faculty personal choice points, of which:
      • 6-6.5 points should be courses from the “core” list (Project Planning and Management, Computer Security, Database Systems, Introduction To Artificial Intelligence, Software Design)
      • At least 15 academic points must be computer science faculty courses
        • this must include at least one project (beyond the mandatory “Yearly Project in Software Engineering” course).
      • The rest of the academic points can be from either list A (computer science faculty courses), or list B (the list of all the other courses that appear in the different programs in the faculty. This includes courses from electrical engineering faculty, industrial engineering and management faculty, math faculty, and various sciences).
    • 10 free choice academic points. Most people spend them on humanities courses.
  • In the mandatory courses, you are required to take:
    • 2 sports courses
    • A total of 8 academic points in scientific courses. One of them must be Physics 1M, while the others can be chosen by the students, provided that in doing so they complete one of the following “groups” of courses: Physics 2MM OR Biology 1 + General Genetics OR Fundamentals of Chemistry + Organic Chemistry OR Fundamentals of Chemistry + Physical Chemistry.
    • If your score in the English section of the psychometric tests (the Israeli version of the SATs) is below a certain grade, you are required to take Technical English. Otherwise, you are exempt (you get the 3 academic points without the grade).
  • Preparatory courses: If you did not study physics in high school you will be required to study the high school version of Physics 1M before you can take this course. Same is true for Physics 2MM if you choose to take it.

Happy reading:

Posted in Academia | Tagged | Leave a comment

Best of Microsoft Research YouTube channel

The reason I like Microsoft better than their “direct competitors” (Google, Apple) is that they have a good balance between making products and creating new technologies. Google has awesome computer scientists and a lot of really good ideas, but they are not very good at turning these ideas into products; they have some successful products (Gmail, Android, Google Earth), but also some spectacular fails (Buzz, Wave, Orkut) or products that aren’t going anywhere (Reader), and up until recently they had some of the most mind-numbingly boring designs ever (thank god that’s over). Apple is awesome at user experience and software design, but they are really boring as a tech company; ever since their pioneering of the digital music business and the invention of the iPhone everything they did was pretty much just taking existing ideas and making a really good version of them.

Microsoft however is like the fusion of the best traits of both companies; despite what their reputation may suggest, their good (even great in some cases) products still outnumber their failed ones, and at the same time they have a huge R&D budget that yields a lot of interesting stuff. Which brings me to this post:

Today I want to highlight some of the cool stuff they have in their YouTube research channel (some of which already made it to actual products), and the following is a list of my favorite videos. Before we get to it though I’d like to clarify something first because I see people not “getting the point” quite a lot on the internet. The point of a research center is not to produce products, it’s to test ideas. None of the research that goes on in Microsoft’s research centers is tied to specific products or meant to have a business idea behind it, and that’s how a research center should be. These ideas are studied and prototyped, and only later integrated by engineers into specific products, often not in the specific way you may have seen them in a demonstration. So please don’t leave comments saying stuff like “when will we see this product” or “I would never want to use it like this”. The LightSpace project is a great example. Would you want to put something like this in your house? No. But that’s not the point. The point is “let’s see what we can do with depth sensing”. Figuring out “what is it good for” comes later (maybe integrating it into research about holograms, maybe there are commercial or military applications, etc etc)

Posted in Microsoft, Software | 1 Comment

Software engineering in the Technion – semester 4

This is the fourth post in my series about studying software engineering in the Technion. If this is your first time reading a post in this series I recommend that you read the introduction post first (go ahead, it’s rather short). If not, I’ll give you a short reminder of where we are:

The fourth semester is in the “second tier” of the degree, which is where we finally leave math behind and concentrate on the essential computer subjects of the degree.

All the courses in this post are mandatory.

Table of contents:

  • Digital Computer Architecture
  • Operating Systems
  • Automata And Formal Languages
  • Algorithms 1
  • File Systems
  • Object Oriented Programming

The fourth semester

cpuDigital Computer Architecture
3 academic points. Computer science faculty course.
Prerequisites: Logical Design, Computer Organization & Programming.
Adjunct (must be studied with or after): Operating Systems.
Course site.

What is it about: The inner workings of modern processors.

About the course: This is another one of my favorite courses. Despite its name it actually focuses on 1 part of computers- the Central Processing Unit (CPU), also known as the computer’s processor.

Modern CPUs are really impressive things and do much more than just calculations. In the course of figuring out how to build faster CPUs all kinds of mechanisms were invented that do above and beyond just crunching numbers, and are frankly quite cool. Here’s a selection of some surprising things you discover in this course:

  1. CPUs execute multiple commands at the same time. At any given point there are multiple commands in different stages of execution (this is called pipelining). Think about this for a moment: what if two currently executing commands depend on one another? How can they possibly execute at the same time? (and they do).
  2. Not only do CPUs execute multiple commands simultaneously, but they actually execute them out of order. This means that when the CPU rearranges the order of commands it must make sure it doesn’t change the meaning of the program.
  3. A common element of programs are branch commands, which look like this:
    if (something) doThis() else doThat().
    A CPU actually guesses which one of these options is going to happen before it discovers the answer by checking the something condition, and it does this with over 90% accuracy.
  4. A CPU has its own private memory, and has built in support for managing the external memory (RAM). That’s right. The calculation unit concerns itself with how memory is managed.
image
This is a die photo of the Intel Core 2 Duo. The big grid area to the left is its level 2 cache memory.
 

When I took the course we had a guy called Lihu Rappoport teaching the course. He worked as a hardware architect at Intel and you could tell that he lives and breathes this stuff. Not only is this exactly what you want from a lecturer, but he also told us the occasional “that’s how they do it in the industry” remarks, as far as his confidentiality agreement allowed him of course.

What is it good for: While this sounds like a course for computer engineers, it actually teaches some things that a systems programmer should know as well. It kinda bridges the gap between the hardware and the software by telling you about what happens to your code as soon as it’s sent to be executed, and it does this without getting “down and dirty” with the hardware. It talks about concepts and systems, not circuitry and logical gates.

tuxOperating Systems
4.5 academic points. Computer science faculty course.
Prerequisites: Data Structures 1, Computer Organization & Programming. 
Course site.

What is it about: The inner workings of operating systems.

About the course: This is another totally awesome course. Not only does it covers all the concepts of modern operating systems (memory management, context switches, scheduling, device drivers, file systems, threads and processes, inter process communications, etc etc), but it also gives you hands on experience with these things by letting you study and make changes to a real operating system (Linux).

Linux_kernel_mapThis is the structure of the Linux operating system.
 

Needless to say this course is quite a juggernaut, and it’s arguably the most difficult programming course in the degree. There’s tons of information to learn, a lot of which can be very confusing (memory management, concurrent programming), you are required to study parts of a huge and complicated system (Linux), you find yourself working without debuggers or convenient IDEs, and the homework assignments themselves are huge and difficult (each assignment is given 3 weeks and is done by teams of 3 students). You definitely feel like your are being thrown into the water here, but the payoff is big as well: you learn a lot, you get to do some pretty advanced stuff (I actually list the things I did in this course on my resume), and personally I felt a noticeable “level up” in my programming skills after doing this course. Also, the whole subject is just incredibly cool and interesting. In fact, I even revisited it in subsequent semesters by taking a Project In Operating Systems course, where I got to implement my own operating system.

pagefaultsThis is how Linux handles the “Page fault” error.
 

I mentioned in previous posts that some courses have a personality associated with them, someone who is known as the teacher for the course. This course has Leonid Raskin, and the thing that he is known the most for is how much effort he invests in helping the students. He actually opened threads in the student union forums dedicated to answering questions about the assignments, which no other lecturer does, and actually visited them frequently.

I want to give a description of what the assignments covered, because I feel that they really are the essence of the course. This is a little self indulgent of me, but what the hell, it’s my blog. I did mention all the important things about this course, so if you feel like this is too technical/confusing just skip to the next course.

  1. First assignment was pretty easy: implement a few system calls (a system call is how programs ask the operating system to do some service for them). It was relatively easy because teaching us how system calls work was almost secondary to teaching us how to search for things in the Linux code, which was the real challenge.
  2. Second assignment was hard. We had to add a scheduling policy to Linux, which is a mechanism that allows us to run multiple programs simultaneously (a computer can really only do one thing at a time, and running multiple programs is achieved by switching between them really fast: the OS lets program A run a little, stops it, lets program B run a little, and so on… A scheduling policy decides the order in which we let programs run, and for how long).process
    There were some late nights in this assignment; We had to do a lot of Googling and poking around, and working without proper debug tools was not easy. At some point we even decided to throw away what we did and start over because of a bad design choice that caused all sorts of strange problems. Also, only 2 of us did any work (you have to really choose your partners well for this course). The end result was good though.
  3. Third assignment was really really really hard. We had to implement User Space threads, a mutex library to go with it, and an I/O library to go with that, and for those of you who don’t know what this means I’ll just say that this is quite a lot of work. It is basically reimplementing a core part of the operating system from scratch. We had to plan the structure that represents the threads, think about scheduling them and switching between them, implement a way for them to communicate with one another, think about releasing resources, and that’s just the threads part of the assignment, there were a bunch of other challenges associated with the rest of the assignment too. threads
    The fact that we had to do this in “user space” (which means that we are not allowed to change the operating system itself) meant that in some cases we had to do all sorts of hackish things to get around the way the operating system is supposed to work. Another thing that greatly added to the challenge was that this was basically an exercise in concurrent programming, and concurrent programming is really hard, especially if you are doing it for the first time.
    We spent many sleepless nights on this. In fact, for a week and a half we did nothing but working on this assignment, skipping all our other classes.
  4. After the third assignment, this one wasn’t so hard. We had to implement a networking protocol between a client and a server. It had its challenges, but wasn’t anywhere near the level of difficulty of the third assignment. Unfortunately this was our lowest homework grade because we were too burned out from the third assignment, but it was still cool to see how two instances of the operating system communicated with one another.

What is it good for: Pretty much everything you learn here is important to know, even if you don’t intend to be a systems programmer, because the concepts are transferable, and in some cases they are actually transferable “as is”. In particular, this is the first time you meet concurrency and synchronization mechanisms, which are things that everybody need to know. Threads and scheduling are also incredibly important. Also… Well screw it, it’s all important to know. Especially if you are a systems programmer.

automataAutomata and Formal Languages
3 academic points. Computer science faculty course.
Prerequisites: Logic And Set Theory For Computer Science
Adjunct (must be studied with or after): Algorithms 1.
Course site.

What is it about: This covers the mathematical models of regular and context free languages, and the theoretical machines that can recognize them.

About the course: It’s a little tricky to explain this course because it’s very theoretical computer science, but I’ll give it my best: imagine that there are different kinds of machines that work a certain way. Their purpose is to read words and figure out if they belong to a certain predefined language. This theory explores the following questions: what can these machines identify (can they identify all languages or only some of them)? What can we tell about the languages that these machines can identify? How are the different models related to one another?

This seems very detached from reality, but the field had some direct contributions to practical computer science, and it’s actually a very nice and interesting theory. It’s very logic-esque, and I really enjoyed it, even though it was the only subject I ever failed in my degree. (It’s not a difficult subject but they sure are pedantic when they check the exams). The TA was kinda cute too, so that part was nice.

What is it good for: Regular languages are actually used quite a lot by actual programmers, though they know the tool as RegExp (short of Regular Expressions). They are useful for doing advanced searches in text, and also for advanced text manipulations. The theory of the “automata” that can identify regular languages became a very popular design pattern for many parsers. Formal languages are the field’s real achievement though. They are powerful enough to describe all programming languages and the theory behind them literally gave us a new, much better way of writing compilers, turning them for ad-hoc-ish, highly complicated pieces of software into much more elegant and modular programs, and also allowed us to automate part of the process of writing them. Also, one important thing you learn from this course is when not to use regular expressions to solve a problem.

pushdownautomataPushdown automata: recognizing context free languages since 1970s (I think…)
 

algoAlgorithms 1
3 academic points. Computer science faculty course.
Prerequisites: Data Structures 1, Combinatoric Theory For Computer Science.
Course site.


What is it about: An extensive look at Graph algorithms.

About the course: While it may seem strange that it focuses mainly on graph related algorithms, there’s actually a lot of merit to the decisions. First, it’s a pretty deep subfield of algorithms, so there’s a lot of ground to cover, and second, graphs are extremely important in computer science. A lot of things in computers are naturally represented by graphs (networks, the structures of file systems, flow charts of code, etc), and as it turns out there are plenty of seemingly unrelated things that can also be represented and reasoned about as graphs (for example, you can describe calculations with graphs). This course covers all the classic algorithms, problems, and tricks of the field, and also has a lengthy discussion about the NP-Completeness of certain problems, which is something that is studied in depth in the course Computability Theory. This is a pretty difficult course, and you definitely have to have a bit of luck to pass the exam (unless of course you really get this stuff) because you are required to not only come up with the algorithm that solves the question, but also prove it, and both things take a lot of time. You pretty much have to write as fast as you can the whole exam.

Justsearch1The two classic ways to search through graphs.
 

What is it good for: Tons of stuff. Let’s leave it at that.

harddiscFile Systems
3 academic points. Computer science faculty course.
Prerequisites: Data Structures 1, Probability M.
Course site.

What is it about: The inner workings of hard drives and the information storage systems that run on top of them (A.K.A. File Systems).

About the course: All data is essentially heaps of ones and zeroes arranged in a certain way, and we need some way to make sense of what this heap of numbers means. A file system is essentially that: it dictates how information is stored on the hard drive. It concerns itself with organizing data into files and folders, and with making changes to this collection of data (creating/deleting files/folders, moving things around, etc etc). Turns out there are all sorts of interesting problems that are associated with this. We need to consider space efficiency, avoid fragmentation of the data, make accessing information as fast as possible, make the system fault tolerant (errors shouldn’t mess up the file system), make it possible to repair the file system if it does becomes corrupt, make it as “concurrent” as possible (multiple programs can access different parts of the file system simultaneously as long as they don’t clash with one another), and there are a bunch of other issues that are too numerous to list.

image
The inode is how files are represented in Unix like systems.
 

Another thing the course covers in great depth is the inner workings of hard drives. What you discover in this course is that hard drives are really problematic things. Not only are they the bottlenecks of the system, but they are also the most likely to fail components (aside from maybe the fans). So many tricks and ideas were invented to overcome these shortcomings. You learn about RAID arrays, which are systems that use multiple hard discs to do the job of one (the rationale here is that if one hard disc fails, we can restore the information from the others. We can’t recover from two failures, but since two failures are rare, we get a pretty good protection). Another interesting thing you learn is that hard discs do a lot of optimization to make reading and writing more efficient. As a result they often lie to you when they say they are done writing the information because all hard discs have internal memory and they first write to it, tell you they’re done, and then write the information to disc later when it is convenient to them.

What is it good for: Following this course you will understand the efficiency considerations behind working with the hard disc, and gain some invaluable insight into fault tolerance and how file system work in general. The information about RAID arrays is also important to know.

oopObject Oriented Programming
3 academic points. Computer science faculty course.
Prerequisites: Programming Languages
Course site.

What is it about
: The theoretical concepts of object oriented programming.

About the course: This is a direct continuation of the Programming Languages course, and it has the same format. In lectures you have theoretical stuff about the object oriented model, and in the tutorials you get to study about and play with various (object oriented) programming languages, specifically- Squeak (a variant of Smalltalk), Java, and C#.

This is a fun course because it’s not too difficult and you learn stuff that’s both interesting and important to know. For those of you who don’t know, the idea of object oriented programming is that you look at the world in terms of objects that talk to one another, whereas an object is a “thing” that has properties that define it, and actions that can be done to it. For example a “Car” object will have the following properties: amount of fuel, color, manufacturer, and possibly others, and the actions you can do to it are: turn it on/off, drive it, repaint it, fill its fuel tank, and so on. This is the most popular programming model today, and it is used everywhere where performance is not utterly critical.

object

The nice thing about this course is that it fills in all the “missing parts” in your programming knowledge up to this point. It talks about multiple inheritance in C++, the different kinds of inheritance, the types of polymorphic relationship between objects (and functions!!), and a bunch of other stuff that I will not mention here because it is unlikely to be meaningful to the average reader. It is also the first time when you learn Java in the degree.

What is it good for: Pretty much the same thing that its predecessor is good for. You have to know this stuff if you want to be able to have an adult conversation on the topic, and the languages that you play with have some interesting concepts (Squeak has no reserved words, Java is important to know, and C# is just awesome).


So that’s the end of the fourth post. Hope you liked it. To find the rest of the series:

Previous entry Index Next entry

Posted in Academia | Tagged | Leave a comment

Who said Apple is more innovative than Microsoft?!

Every time the topic of innovation in (computer) technology comes up it’s always the same tune: Apple is innovative and Microsoft is not, and apparently hadn’t been innovative for many years. I won’t deny that Apple has some good products or that they did some innovative things in the past, but I can’t help but wonder what the hell is it that everybody is smoking? Even if we ignore the fact that Apple hasn’t done anything innovative since the iPhone (everything they did since then was mostly incremental improvements, and no the iPad doesn’t count; It’s not the first tablet, the OS isn’t new, the UI isn’t new, even the form factor isn’t new, they just came up with the first successful implementation. So what exactly is innovative here?) saying that Microsoft doesn’t innovate is just plain wrong, and frankly I’m quite shocked that I need to spell that out to people (well, OK I’m not literally shocked, I actually have a theory about it but I’ll leave it to the end of the post).

The following is a recollection of some of Microsoft’s innovations in the past 5 years, with some tidbits about related research thrown in between (because the academic basis behind the products can be just as innovative as the product itself. I’ll limit myself to research that actually influences real products though). Just to make sure we are all on the same page here, I’ll supply the dictionary definition of what the word ‘innovation’ means, posted here specifically for the benefit of Apple fans who seem to redefine the term whenever it suits them:

Apple fanboy: Apple did something new, therefore it is innovative and others aren’t.
Non-biased-person: X also did something new.
Apple fanboy: Yes but Apple did something new that is also cool. See, it has to be cool and sexy to count as innovative.
Non-biased-person: Well, Y did something that was both new and cool.
Apple fanboy: Yeah, but Apple did something new and cool that also gained traction and changed how we view <whatever>, and Y didn’t, so Y still doesn’t count.
</straw man>

Definition of INNOVATION
in·no·va·tion noun \ˌi-nə-ˈvā-shən\
1: the introduction of something new
2: a new idea, method, or device : novelty

Microsoft innovations in UI:

Windows Phone 7 and its Metro UI

Windows Phone 7 is the first new approach in UI design in the mobile arena since the iPhone. What Apple did with the iPhone was truly innovative, in more ways than one. But since then we had nothing but imitations.

iphone-iphone-3g-comparison

Everybody basically do an iPhone-esque “palette of apps” kind of model in which you have an app for each task and the job of the OS is to be an app repository. So yeah, Android and MeeGo and the rest of the gang all played around with custom home screens that weren’t a collection of apps, and in some cases there was even more than one such screen for various utilitarian reasons, but at the bottom line these special screens weren’t much more than a custom skin around the same operational model of an app per task. They simply brought certain tasks to the forefront.

Windows Phone 7 is fundamentally different. Instead of fragmenting information and functionality across different sections (apps) we have an emphasis on consolidation, which is done via theme based “hubs” where related information and functionality lives. For example, instead of having a separate app for phone, SMS, Facebook, and contact details, you have the “people” hub which aggregates all the available information about your contacts from multiple sources (phone book, social networks, email, etc.) and allows you to do all of the above without leaving the hub area (as an exception to the rule you do have a dedicated phone and SMS tile in the home screen because these are just very common actions, but aside from that all features are accessed via hubs). This is an information centric design, rather than a function based design, and as it turns out it’s a very fruitful idea. The way you navigate the hubs in itself is new; it’s not a collection of screens but rather one long continuous screen that contains different sections (usually 3 or 4) and you swipe to the side to go to a different section. You can see the 6 hubs of Windows Phone 7 below (click to see a bigger picture).

Marketplacehub_US_web

The way it is designed is also very interesting. Windows Phone 7 essentially tries to minimize the number of UI elements and instead use the content itself as the UI. Just look at any hub. There’s very little in it that is not part of the content, and even when something is a more traditional UI element it blends stylistically really well with the content. The “button” that brings up the ABC selection that lets you jump forward in your contacts/app list also acts as one of the alphabetic indexes in said lists. The backgrounds of the hubs come from their content- in Pictures it’s your picture library, in Zune it’s based on the bands/serials you have in your collection, and so on. The tiles on the home page are not static icons but get updated based on the information they represent, be it your X-Box Live avatar on the Games hub tile, or number of emails in the Outlook tile. Essentially, the content on your phone is the tile icons.

Another clever use of content as UI is the way it gives you navigational hints. You know that there’s more stuff on the side of the screen because you see it slightly perturbing from the side. You know how to unlock your phone not because there’s an arrow that shows you what to do, but because when you touch it, the lock screen bounces, revealing some of the home screen behind it, and thus showing you how to move it to unlock your phone.

The following videos demonstrate a bunch of these elements. The first is a 3 minute “commercial”, while the other two are more in depth demonstrations.

Short commercial-like video

Between the design, the UI and the very impressive interoperability and aggregation features that I didn’t talk about because I wanted to focus on UI at the moment Windows Phone 7 is pretty much the most original piece of work in the mobile space since the first iPhone appeared.

The Ribbon UI

The Ribbon was originally introduced by Microsoft in Office 2007 to solve a problem that plagued the Office suite for many years: an increasing complexity that the standard menu based UI failed to mitigate. I already wrote a long post about why I think it’s great and how it is an improvement over menu based UI, so I’ll only give you the TL;DR commentary about it:

ribbon

not only was this a highly ambitious endeavor- an attempt to replace a UI paradigm that has stood the test of time for many years- but it was also one that required a great deal of creativity and thinking outside of the box, essentially going against years of habit and convention. It’s pretty much a textbook example of what innovation is, and the end result was that they managed to fix most (if not all) of the things that were wrong with Office.

Research: voice, movement, and Natural User Interfaces

User interfaces aren’t really all about buttons and menus though. User interface is any way that technology lets you interact with itself, and we are certainly getting into an era where touch screens are becoming the norm for cell phones, and speech recognition is certainly not new, but there’s still a long way to go before interaction with technology becomes transparent. As Microsoft’s Bill Buxton explains in the following video, making a natural user interface is not about perfecting a certain input method like touch screen UIs or make voice recognition less of a pain, but about learning to merge all these things in a way that makes interaction of intangible things… natural.

What Bill Buxton is talking about here is a relatively uncharted territory, and it’s the kind of thing Microsoft does all the time. While Apple is screwing around with making their iPhone white and finding a way to eliminate yet another button from some UI screen to make it more simple, Microsoft is trying to figure out the future of how we interact with technology.

Of course, it’s very possible that Apple has some research in that area as well, I’ll be disappointed by them if they don’t, but you don’t see it from their output, unlike Microsoft. This natural user interfaces stuff is not all talk; these things have been applied to actual products by Microsoft, which brings me to the next segment:

Innovative sci-fi tech from Microsoft:

Microsoft Surface

microsoft-surface2

Microsoft Surface may seem like a giant touch screen device at first, but it’s actually much more than that. First, it’s not really just a touch screen device. One of the goals of the device was for the user to be able to place items on the screen, and have it recognize the items and allow the user to interact with them. The above image is a great demonstration of that. Do you see how there’s a UI circle around that iPhone and attached to those dice? These are not UI elements with items placed on them. These are UI elements that are generated in real time around items when you put them on the surface, and the device uses a setup of cameras with image recognition to distinguish between different types of items, as well as wireless connectivity that allows it to communicate with electronic devices that support it, so you can manipulate them via the surface. Surface 2.0 took this a step forward and replaced the cameras with a technology called Pixel Sense, in which every pixel on the screen acts as an infra red camera, thus allowing Surface to see what’s placed on it without traditional cameras.

Second, the amazing part about it is the level of interaction and collaboration it provides which really transcends anything traditional touch screen devices did so far, and is a prime example of a Natural User Interface in action. Take a look:

One thing that stands out about this video is how natural it is for the two users to interact together with the content. They show stuff to each other, pass around contacts and images, and so on.
This is a tech demo of Surface 2.0 that covers some of the technical highlights as well as some of the applications.
This one covers similar ground to the previous ones, but shows a bunch of cool stuff more in depth.

Ultimately, the cool thing about Surface is not really the technology behind it, but the things it makes possible.

Kinect

Kinect is an X-Box addon that seemed to have come straight from some science fiction movie. It uses a combination of infra red cameras, motion detection, facial recognition, and voice recognition to allow you to control the in-game characters on the screen by simply moving your body (and some times talking too)! You move- and the characters on the screen move with you. You step into the frame, and Kinect recognizes you and logs you in. You want to pause a movie- just say “Kinect- pause”. There are no Wii-like sensors, no devices attached to your hand, just an infra-red camera and some very advanced computer science, and the result is truly magical. Not iPad magical, Narnia magical.

Kinect is the only gaming addon I know of that managed to sell well (actually setting a new world record as the fastest selling consumer entertainment device, beating both the iPhone and iPad) despite having a poor gaming selection (at least on launch). People just seem to find it really cool, and for good reason:

This guy did a fantastic demo where you can really see how the Kinect responds to movement and how the games are.
The guys from GameSpot interacting with a tiger in Kinectanimals. It’s very hyper-reality kind of thing.
Coming up: Avatar chats. Kinect will track your facial expressions too

 

Aside from the tech being innovative, the way it’s controlled basically forces the UI to be innovative as well, because many things that we learned to do well with keyboard and mouse operated programs don’t really work here anymore. We need to figure out new ways to do discoverability, and new ways to do intuitive design, and we still need to learn how to leverage this cool new trick we suddenly can do- use the body as the controller, and discover what is natural for it and what isn’t. This Ars technica article about the makers of Dance Central for the Kinect illustrates this really well.

Kinect is definitely opens up the door to endless possibilities, and I can already see this pioneering the kind of concepts that will become mainstream over the years as this tech matures (Microsoft is already working on integrating it with Windows PCs). But arguably the coolest thing about the Kinect is not even its potential as a consumer product, but its potential for doing freaking awesome things. One of the reason it sold well was because it was a very hot item amongst programmers and academics, who jumped on the SDK and started experimenting with making cool things. Here’s just a small selection:

Lots more in kinecthacks.net

If this is not innovative, nothing is.

Research: Microsoft Tech Fest 2010 and physiological sensing

Every year Microsoft does a demo fest that showcases their coolest research. You can get an overview of last year’s fest in this Ars Technica article, but what caught my eye was the physiological sensing gizmo. It’s another example of very science-fictiony tech and while I don’t think it’ll make it as a product I’m sure some of its ideas will find their way to other products eventually.

(Another video can be found here)

Microsoft’s innovations on the web

Bing

bing

Just like the mobile phones realm, the search engine world has become very stale. Sure, Google had all kinds of special search features like typing in a math equation or asking for currency conversion, but it was overall pretty static. Microsoft was the one who shook things up. When they decided to compete with Google they didn’t just make a search engine that gives good results (in fact, I’m not entirely sure they made a search engine that gives good results, at least not in my short experience, but I heard some good reviews too) but they took it a step forward and made a system that tries to infer what the user is searching for and filter out the most relevant entries. If you search for a currently played movie, you will get show times, trailers, and reviews at the top. If you search for something that recently got on the news- you’ll see news listings at the top. This is what they call fighting information overload by bringing the results you probably wanted to the foreground.

A lot of the new stuff Google added in the past year has been a direct result of Microsoft adding feature to what everybody assumed was a “good enough” traditional search experience. What I also like about Bing is that “picture of the day” homepage that shows a different picture each day. It might sound silly, but I heard of some people who visit Bing each day just to see what the new picture is, and it’s a great way to market yourself.

Windows Live

While many people are trying to create the next Twitter clone (look! We’re like Twitter but we actually have features!) Microsoft correctly realized that nobody wants yet another social network. Our digital lives are already enough of a challenge to handle- multiple email accounts, multiple social networking accounts, multiple calendars, multiple computing devices in various forms; adding yet another social network doesn’t add value, it detracts from it. So while Windows Live may look like a social network at first sight (you do have a profile and can network “internally” with people), it’s actually something entirely different, and does something very ambitious: it tries to consolidate all of your digital life, both online and offline, into one service.

windowslive

This is something that I’ve only seen Microsoft do. Sure, other companies do similar things (Google’s web apps do interoperate, and we do have stuff like RSS feeds and cloud based back ups), but nobody does this to the extent that Microsoft does. While everybody are focusing on the web as a platform in its own right (even Google’s fork of Linux- Chrome OS, seems to be all about the web despite being a “desktop program”), Microsoft is looking for ways to make the web and the desktop complement each other, and that’s what’s so unique about Windows Live.

Windows Live consolidates your social network updates into its own “wall”, and allows you to interact with all of them from one location (you can do Facebook chat via Windows Live for example). Then it takes this information (contacts, social updates), and brings it to the desktop based Windows Live Essentials suite of apps. Your social updates appear on Messenger, and the same web based chat is available in Messenger too. Your contacts from both email, social networks, and IM clients are all synchronized between Windows Live the web service and Windows Live Essentials; If you have a contact in Live Mail, you have it in Hotmail as well. Windows Live offers free storage (SkyDrive) that lets you sync folders on your desktop with Windows Live Mesh, and it is also used for web galleries uploaded with Photo Gallery, and office documents explicitly uploaded or edited/created with the web based Office apps which integrate with Hotmail. Windows Phone 7 is also tightly coupled with Windows Live. The following graph describes the relationships between the different services. Click to enlarge.

windowslive2

I don’t know of any other online service that attempt to combine mobile phones, social networks, desktop applications, and productivity elements like Office and email/calendar in one place, and it does quite a good job at it, even if it’s not a perfect implementation (navigation-wise it could use some rethinking, and it’s definitely not the best cloud storage service or the best web email client). It’s clear to me that in many ways Microsoft is still discovering how to make this a cohesive service, but it’s the ideas we’re talking about today, and as an idea it definitely brings a new philosophy to the table.

So if Microsoft innovates, why is it not perceived as such?

One reason for this is fragmentation. Apple is primarily a consumer electronics company. Everything they do is targeted at the home user, in particular the kind that is more into gadgets than general purpose computing; even their general purpose computers are marketed at those types of people. Microsoft’s output is significantly more broad, and if we look at the kinds of innovations I presented here: we have Office UI- that’s business and office stuff, we have Surface which is not even available to consumers, we have Kinect which is technically a home user device but is still in a niche category, albeit a big one; not every home user is a gamer. Bing and Windows Live have much more visible competitors, and quite frankly not everybody will get what’s special about them; You need to be a web savvy user to really feel the difference. Many people are casual Googlers and often stick to very specific things on the web. Only Windows Phone 7 is a bona fide consumer device. Now, let n be the number of innovations people need to see from a company to notice that “hey, they pull awesome shit all that time! They must be innovative!”. To reach n for Apple you need to fit only into one category- gadget consumer. To reach n for Microsoft you need to be invested in multiple domains. Most computer users are casual users and are extremely limited in what they do/know, so it’s basically a “I get excited only about stuff that’s important to me” kind of thing.

Another reason is the domain itself. Consumer electronics- Apple’s domain- is where all the hype is right now. Office tools are not sexy, search engines are not hip, it’s all about gadgets these days; technology you use on the go… and when you look at the situation through that point of view, you quickly realize where the perception I tried to debunk here comes from. It started with OS/X, which came in at the time when Windows XP was getting stale. It was a more modern OS, which looked better and fixed all those usability quirks that plagued XP. With the use of some truly brilliant advertising they captured a lot of mindshare, and with OS/X and iLife they scored a win in the battle for home user appeal. Microsoft’s answer was Windows Vista, their biggest disaster after Windows ME. Yet another score goes to Apple. “Apple did something cool and Microsoft couldn’t top it”.

Around that time the iPod line of products matured, and it was a domain where Microsoft had no presence. Like I said, Apple basically pioneered the digital music business, and when Microsoft entered the market with the Zune it didn’t manage to displace the iPods. The reason here was more related to bad timing (mp3 players being cannibalized by smart phones) and a not aggressive enough marketing (specifically lack of availability outside of US/Canada hurt it), but it didn’t really matter to people. It also didn’t help that the first generation was a “we have to get something out” product and as such wasn’t much to write home about (looked like the iPod a few generations back), and by the time Microsoft released the 2nd generation, which was truly fantastic and on par with the iPods (not to mention the Zune software blows iTunes out of the water) many people lost their short attention and went back to iPods.

When the iPhone appeared it destroyed all competition (as far as being the best that is). Meanwhile, Microsoft was still trying to force the desktop metaphor into a tiny cell phone screen with Windows Mobile. It took a while for them to admit that this was a failed attempt, and only now do they finally have a true counter to the iPhone, many years after its launch.
Tablets are yet another area that is new, and so far Microsoft doesn’t have a good offering for, and while I think the iPad is nothing special and is the most boring way possible to implement a tablet, it still generated hype. It’s still new to people who are not tech geeks. It still brings them the simplified experience that made Apple such a success lately.
I think the following image summarizes the situation pretty well (you can enlarge it):

perception 

The situation today

So yeah, maybe I shouldn’t be surprised that Microsoft is perceived as not innovative, but it’s still very far from the truth. But the reality is not really grim for Microsoft: they still have a lot of technology that Apple doesn’t (X-Box/Kinect, some impressive web stuff), they now have a good counter to the iPhone, their attempts to counter iLife with Windows Live Essentials are very promising, and while they don’t have a public tablet strategy yet I do believe that this is one of the major focuses behind Windows 8 (Windows 7 apparently wasn’t a very good fit for tablets) and when the Windows 8 tablets begin to arrive, I’m sure that they would be way more inspiring than an oversized iPhone that acts mainly as a content consuming device.

Microsoft has strong competitors, and that’s fantastic. But people shouldn’t forget that those competitors have enough to be worried about too. While Microsoft and Google have tons of web stuff, from search engines to email to cloud computing to content aggregation, Apple hasn’t really done anything worthy in that realm. They have the locked-down-behind-a-pay-wall MobileMe, which is sort of like their attempt to counter Office: it can do some stuff, but is nowhere near defeating the competition, and now they have Ping, which isn’t very likely to drive them forward. They better have something huge up their sleeve, because the web is the future in many ways, and both Microsoft and Google get it.

Google on the other hand, has Microsoft’s desktop native software empire to fear. Google get the web, but it’s their limitation to web only technology that is detrimental to them. We are not ready to do everything online yet. HTML/CSS design is still not as versatile as what you can do with a compiler. Microsoft’s attempt to create true, smooth interoperability between desktop and web applications may still be in development, but it’s a powerful concept and the one that is most likely to challenge Google in anything they do.

So in conclusion: stop spewing bullshit, sit back and enjoy the competition.

Posted in Microsoft, Software | 2 Comments

iTunes annoyances

I’m eagerly waiting for Windows Phone 7 to finally come to Israel. The primary reason is that I’m very excited about the interesting stuff they did with it and really want to try it, but there’s another, secondary reason that isn’t as important but will make me so much happier: I want to get rid of iTunes already.

iTunes may have started as a simple and elegant player, but by now it’s pretty broken, in many ways and in all the most annoying places, and what’s really aggravating about it is that Apple just doesn’t seem to care. So many of the problems I’m going to rant about are easily fixable, and in many cases shouldn’t have been problems to begin with, and every time another update arrives I think to myself “well, maybe this time they’ll finally fix everything”, but no. The result is a program that could’ve been great, but instead is the most complained about program in Apple’s iSoftware collective, and quite frankly is the most un-Apple-ish program of a company that prides itself on simplicity and attention to detail.

In no particular order, this is why I hate iTunes:

It’s slow

iTunes is ridiculously slow for a media player. Long start up is one problem, but it’s the bad responsiveness that really gets to me. Every time I plug my iPhone into the computer I know I’m going to have to wait a few seconds before iTunes registers anything that I’m doing, even if I just want to open my music collection. It gets better once it’s done backing up the phone, but still feels somewhat sluggish and there are still unexplained hangups.

As a software engineering student I can understand that getting tech right can be tricky, and I can be very forgiving when I see that the deficiencies are the result of the difficulty of the problem. But in this case it’s not a matter of the difficulty of the problem, it’s a matter of Apple being incompetent. How do I know that? Because the Zune software is much more graphically intensive and yet outperforms iTunes all the time on my computer, and if Apple still can’t do updates the smart way (their idea of a software update is an automated redownload and reinstallation of the whole program) then it doesn’t surprise me they can’t figure out what efficiency means. I guess developing only for high end laptops can spoil you.

The way it handles metadata is terrible

iTunes has so many metadata fuck ups that this is the thing that gets on my nerves the most. Take the following picture for example (you can enlarge it):

itunes1

You will notice that the tracks for Belus and Burzum are misarranged. Why is that? It knows they belong to the same albums, they are all numbered, and yet it doesn’t sort them correctly. Want to guess why? In Burzum it’s because track 6 doesn’t have “Disc number 1 out of 1” filled in. What kind of weird ass sorting algorithm could they possibly be using there that takes a collection of numbered tracks that it knows belong to the same “entity” (Album name) and fails to sort them in the correct order unless you explicitly tell it that there’s only 1 CD (like it matters)?!
(Don’t answer this. I know why it does it and it’s stupid).

You know where this gets really annoying? When you sync these tracks to your iPhone and suddenly realize they are playing in the wrong order. And of course I can’t fix this via my iPhone because that would require another UI element that will confuse my poor technically challenged grandma, so I have to live without it too, changing the songs manually as I go instead of letting the player do this for me. (yes I can also create a playlist. Do you think I want to have to do that?)

Its lack of support for multiple artists/composers is also a source of great annoyance to me. All of Microsoft’s media players know that “Composers: John 5; Twiggy Ramirez” means that there’s a distinct composer called John 5, and a distinct composer called Twiggy Ramirez, and that they collaborated together. So if I ever wanted to get a list of all songs where John 5 was a composer (for example- for making a John 5 playlist), all I need to do is go to the single John 5 entry and all those songs, with or without collaborators, will be there.

itunes2

iTunes is not that smart apparently. If you give it “John 5; Twiggy Ramirez” as composers it’ll go ahead and create a new artist called “John 5/Twiggy Ramirez” on top of the existing John 5 and Twiggy Ramirez artists, and will associate the track to that new “artist”. The result? FRAGMENTATION (enlarge to see the problem at its full glory).

itunes3

…and if you think the single “John 5” entry on iTunes consolidates all the tracks- it doesn’t. It only shows tracks where John 5 was the sole composer. So now to make a John 5 playlist I need to go to each one of those entries, in a constant back and forth between the search results, and and hunt down the tracks I need, all for no good reason.

The iPhone also inherits this problem. Since you can’t sort your iPhone’s music by “album artist”, only by “artist”, then for example if you have a duet between the artist Burzum and Euronymous on Burzum’s self titled album (so this track’s artist will be Burzum/Euronymous), that track will not be in album Burzum when sorting by the artist Burzum, and instead will end up in an entry for the the new “artist” Burzum/Euronymous.

itunes4
So if you want to listen to the album from start to finish you have to either access it via artist Burzum and manually switch to that track when the time comes, or you will have to hunt down the album when sorting your collection by albums, which is less convenient than sorting by artists. Thanks Apple.

Oh and speaking of sorting by artist, how do you like this little gem?:

IMG_0807[1]

Thank God I have my iPhone to notify me of trailing whitespace in my metadata. After all, we don’t want the two different bands “Dark Funeral” and “Dark Funeral ” to be mixed up. They play totally different genres. One is Black Metal and the other plays children’s songs.

Too lazy to fix this for me, iTunes?

The brain-dead way it handles unknown tracks

Suppose you have a folder with files, and you decide to import it to iTunes. God help you if one of those albums lacks metadata. You will end up with an “Unknown Artist” entry for every damn track.

itunes 6

WHY?! Who the hell thought this was a good idea?! Media Player at least aggregates all these unknown tracks to the same “unknown artist” entry, and then you can use the “File path” column to quickly make sense of the mess. In iTunes I have to enter each “Unknown Artist” entry and change the data track-by-track. And you’d think iTunes could at least be smart enough to know that if you rip a CD and it doesn’t recognize the artist, all songs are probably from the same artist (simple statistics of number of LPs vs various artists collections). But no, you still get one entry of “Unknown Artist” per song. Really, Apple? You call that user friendly?

Its weird and unnatural selection mechanism

Remember the time when holding down shift, pressing down to select some tracks, and then pressing up did not deselect the last selected track but instead selected additional tracks above the current selection?

So they fixed it in song selection, but apparently it hadn’t occurred to them to do the same in the artist view. So suppose you just imported a folder with no metadata in the files and got 500 “Unknown Artist” entries in your library, and decided to delete them, use mp3tag and then reimport them. You better be really careful if you plan to select these folders using the keyboard, because this non-standard selection scheme makes it impossible to fix your selection if you selected too much. In fact, it’s way worse than that. The selection scheme makes no (good) sense. Go ahead and experiment with it. It sort of feels like playing Snake, except that you quickly realize that it only acts like that some of the time, and that the left key is really bound to the “up” side of the snake, and the right key to the “down” side. It’s overall confusing.

itunes5
Snake: the iTunes version. FYI, original selection started at Venom. Yes, this is done with a keyboard.

While most file/music/whatever managers remember your position in the collection of items after you do deletion (so when you press to the right for example it would select the item to the right of the selection you just deleted), iTunes doesn’t remember anything, and pressing right will make you jump to the top of the artist tiles, forcing you to rescroll down to select what you wanted to select.

It doesn’t watch the file system for changes

Every media library organizer that respects itself can do this, why can’t iTunes do this too?! This was the feature that made me figuratively fall in love with the latest Media Player. You drop some files into your music folder, and it automatically finds them and adds them to the library. You change some metadata in one of those files- Media Player updates itself automagically. It’s one of those features that once you have them you don’t ever want a media library organizer without it. …and iTunes doesn’t have it.

This to me says that Apple doesn’t care that some people don’t get all their music from iTunes. If you buy music from the iTunes store it’ll update its library for you automatically, but if you’re not an iTunes customer- screw you, you’re on your own. So now for every new album that I get I have to manually import it to iTunes, and if I move something, I have to update iTunes manually again. If you change some metadata outside of iTunes, it at least will recognize this when you use it to access said file, but it doesn’t have a very good track record where album covers are involved. Usually, album covers inserted in Media Player (fully committed to the hard drive) fail to be detected by iTunes, and I have to reinsert the album cover via iTunes itself. Again.

All of this is stupid and unnecessary hassle. In that respect iTunes is behind even some of the crappiest Gnome media players that litter the Ubuntu software center, and there is some serious crap in there.

UI lameness

Why the hell is burning a CD achieved via a playlist? How is that natural? Not every CD that I want to burn is a song collection that I want to listen to in the future. Why do I have to go through the hassle of creating, and then deleting, a playlist? And you know what else sucks? I can’t create a playlist without it taking me to the screen that explains what a playlist is, so now I have been thrown out of what I was looking at and have to go back.

What about all those checkboxes next to every song? What are they supposed to be about? Am I supposed to guess? Will I win a white iPhone if I do? (And don’t write to tell me what they are- I know what they are. But they have no explanation in the UI. You have to dig in the help file or experiment to figure out their function).

By the way, the iTunes UI really hasn’t aged well. Have you seen the Zune software? It’s beautiful. Blows out of the water everything Apple ever did. iTunes looks like something from the 90’s. Still. And in all the years of accumulating features it has been extended in the most naïve, ungraceful way. Got a new feature? Throw in another section to the side bar! I agree that there’s an ease of access element here, and that this scheme is not bad per se, but there’s an invisible line where it stops looking good, and iTunes has crossed it. It just doesn’t feel elegant anymore (and don’t get me started on the iTunes store).

Crippled if the iTunes store is not available

So I get it, the anal record industry wants you to get a license for every note of every song you make available in the iTunes store. But why can’t I access the related data? In Media Player I can right click an album and ask it to fetch the album information from the web. I get metadata, covers- the works. iTunes doesn’t offer that to me. Sure, it has the option to fetch album artwork, but I can count on 0 hands the amount of times it found anything for my library. I think it just doesn’t work because I’m not in an iTunes store enabled country, and that’s dumb.

What about Ping? Why can’t I use Ping to network and follow artists? Because “the experience will be crippled seeing as I don’t have access to the tightly integrated iTunes store”? My experience is already crippled.
In contrast to this, Microsoft’s Zune social network is available to everybody.

But maybe I should leave this alone, since I’m not sure why I would like to get in to begin with.

I think this about sums up what I hate about iTunes.

Do you hate iTunes too? Feel free to tell me about it.

Posted in Apple, Software | 7 Comments