A functional introduction to CEDET – EDE

CEDET is a collection of elisp packages for Emacs designed to turn it into more of an IDE. While it does have the ability to better bridge the gap between editor and IDE, it can be an enormous pain in the ass to figure out; it is “documented” in a rather haphazard way, and resembles an API more than a UI, and as anyone who had to work with uncommented code knows, this is not exactly a recipe for fun.

There’s an article called “A Gentle Introduction To CEDET”, which is pretty good and helped me out in understanding the cacophony of features that is CEDET, but it certainly wasn’t enough. I also spent time reading the manuals for the various sections and had to play with the features before I finally began to comprehend how to get CEDET to do the things I installed it for, and the main take away from this experience was that CEDET has horrible usability, but it’s the best tool we have for enriching Emacs with IDE features, so someone better write an article that addresses CEDET from a functional point of view (i.e. I want to do X) rather than the “understanding the internals” point of view that too often GNU projects take. Today, I’m going to cover EDE, CEDET’s project management module.

Project management

Setting up project management is the very first thing anyone should do to use CEDET, because many of CEDET’s tools use the project definitions to get necessary information. Project management is handled by the EDE component, and supports the following features:

EDE features

  • Manage makefiles: add/remove targets, add/remove files to targets.
  • Project configurations (i.e. debug build, release build)
  • Create distribution file
  • Initiate building, debugging, or running of the project.
  • Viewing and updating project metadata (this includes anything from giving the linker special options to choosing a compiler to updating the project’s version number). This feature is meant for getting down and dirty with EDE, but there’s a GUI for this, so you don’t have to mess around with elisp. Note: this section is described near the end of the post.
  • Searching for files in a project
  • Viewing project tree organized by targets
  • Managing the project’s web site. You can define a “web page file”, a host address, an FTP server address, and a mailing list address for your project. EDE includes the feature to update the project’s website and upload the distribution.

To turn EDE on, place the following code into your .emacs file:

(global-ede-mode t)

When EDE mode is on, and you visit a project, you will see a “Project” option appear in the menubar. Many of the commands I’m about to describe can be accessed via that dropdown menu.

The EDE world view

There is one important thing to understand about EDE, and that is the way in which it looks at your projects.

EDEprojectmodel

In EDE’s view, a “project” is files within a folder + their makefile. If you have multiple folders in your codebase, then each one of those folders needs to be defined as a “project” as well, and these “projects” are considered to be subprojects of the top level project.

It’s worth mentioning that EDE is aware of things that happen down the folder hierarchy. You can be at the top folder, initiate a file search, and it will search the entire hierarchy. When you add a subproject, the makefile for the top project is automatically updated to build the subproject as well.

Supported EDE use cases

The question you need to ask yourself when you decide to define a project is which one of the following scenarios fits you:

  1. I’m starting my project from scratch, and want EDE to manage it for me.
    This mode of operations is the most feature complete because as you write your code you set everything up the way EDE wants it to be. This option lets you create either Makefile projects or Automake projects.
  2. I have existing source code that EDE knows how to work with.
    This mode of operation is good for Automake projects, and 2 specially supported projects: Emacs and the Linux kernel. In these situations you need to just go to the source code and EDE will identify everything automatically. This mode of operation is less feature complete, in part because it assumes that since you have existing makefiles, you probably want to keep managing them yourself.
  3. I have existing source code that EDE doesn’t know how to work with.
    In this mode of operations none of the project management features will work, but you can still use EDE to define your code as a project. The reason for doing this is so that some limited amount of metadata will be available for other tools in CEDET. This mode of operation will require you to write elisp code, but you can have a minimal setup with very few lines of elisp. I’ll explain how to do this easily further in the post (there’s a template). Since C/C++ projects with existing Makefiles are common, EDE has a special “project mode” for them.

Note: EDE now supports Android and Arduino projects too. These features appear to be relatively new, and I will not cover them in this tutorial.

With that in mind, let’s go over the 3 use cases…

EDE with new projects

The EDE documentation has a quick start section that is worth looking at for a short example of how to work with EDE for new projects. The general gist is as follows:

Suppose we have a folder hierarchy as depicted in the picture above: a folder with c files and a header, and two subfolders, also with c files and a header in each (currently there are no makefiles or Project.ede files). You need to do the following:

  1. Open a file in the top level folder, and type M-x ede-new. This will ask you for a project name and type: make or automake. This creates a Project.ede file in your directory.
  2. Type M-x ede-new-target or C-c . t, and it will ask you for a name, target type (see below), and if you want to add the current file to that target.
  3. Repeat step 2 as needed to create all the targets you want.
  4. Open a file in the current directory, and type M-x ede-add-file or C-c . a and it will ask you which target you want to add this file to.
  5. Repeat step 4 as needed to add all the files to the necessary targets.
  6. Repeat steps 1-5 for each subfolder, and then their subfolders, and so on.

Supported targets for ede-new-target:

The command ede-new-target requires you to specify what kind of target it is. The following is a list of possible targets and what they are meant for (thanks to Alex Ott from stackoverflow.com for the list).

  • archive – object code archive/static library
  • auxiliary – handles auxiliary files in project – README, *.txt, etc. When I played around with this type of project the header files I added to a “program” target ended up here as well. It looks like this is for files that don’t get compiled.
  • emacs lisp – compiles Emacs Lisp code
  • emacs lisp autoloads – target to create file with Elisp autoloads
  • info – handles texinfo files
  • miscellaneous – target that allows user to compile other parts of project, using user-provided Makefile (one that is not generated by EDE)
  • program – builds executable program
  • scheme – target to handle Scheme (guile) source code
  • semantic grammar – target to correctly build Semantic’s grammar files, both for Wisent & Bovine. This target is defined in Semantic, not in EDE
  • sharedobject – target to build shared libraries

Supported features:

The following is the complete list of features that EDE supports. When working in this manner, they should all work or be supported:

Create target: Creates a target for your makefile, and possibly adds the current file to it.
Invocation: M-x ede-new-target or C-c . t
Remove target: Removes a target from a makefile.
Invocation: M-x ede-delete-target
Add file to target: adds the current file to a target of your choice.
Invocation: M-x ede-add-file or C-c . a
Remove file from target: removes the current file from a target of your choice.
Invocation: M-x ede-remove-file or C-c . d
Building: build the current project or target. You can access more target building options from the menus.
Invocation: C-c . c for current target, C-c . C for entire project.
Debug target: Run a debug session on current target.
Invocation: M-x ede-debug-target or C-c . D
Run target: Run the current target
Invocation: M-x ede-run-target or C-c . R
Create distribution file: Creates a tar.gz file with your code in it.
Invocation: M-x ede-make-dist
Locate files: search for a file in your project.
Invocation: M-x ede-find file or C-c . f
Speedbar: Open a project hierarchy navigation side buffer that shows your files grouped by targets.
Invocation: M-x ede-speedbar or C-c . s
Make distribution: This command will create a tar.gz archive of your project.
Invocation: M-x ede-make-dist
Upload distribution: This command will upload the distribution file to the FTP server you define. Using this feature will require you to define the FTP server’s address via the customization feature, which is covered later.
Invocation: M-x ede-upload-distribution

A personal note:

Having played around with EDE for a bit my impression of it is that it’s cute, but not overly useful. While the initial creation of targets is kinda nice, the moment you decide that you want to do something more you’ll quickly find EDE getting in your way, because it’s nothing more than a layer of (bad) abstraction over the Make system. I’m saying “bad” because an abstraction should mean that you create a new, simpler “language” that the user can use without knowing what is underneath it, but in the case of EDE it doesn’t hide any of Make’s complexities from the user, but rather gives you a different way to specify the things that you want to do, and probably already know how to do by hand anyway (not to mention that unless you know how makefiles work, you’re not going to be able to make any sense out of the customization buffers of your targets).

For example, suppose that you need to compile your code using special flags (-Wall) and link a bunch of libraries (-lm). In EDE this means opening the target’s customization buffer and searching for the option to specify libraries you want to link, and flags you want to pass to the compiler. If you already know what these options mean, you already know how to type them in the Makefile and where, and it would be much more natural to just do that instead of figuring out what’s the “EDE way” of doing the same thing.
What makes this even worse is that the way you specify some options is by overwriting the default values of Makefile variables that EDE generates, and how do you know what these variables are and how they are used? By looking at the Makefile of course! Yes, some of them are pretty standard and once you learn them you don’t need to check the Makefile anymore, but the point is that if you need to reference the internal structure of the Makefile then what kind of an abstraction is this? How is this helping us not mess around with Makefiles (which is presumably the reason we want EDE to manage them for us)??
And if this wasn’t enough, there’s also the myriad of Make features that EDE doesn’t support, which is a headache in its own right if you want to do anything remotely non-trivial. This includes basic stuff like initializing a variable using ?= for example. The EDE workaround for this is to let you define files to “include” into the Makefile, which basically means “write all of that stuff in your own Makefile, and tell EDE to include it in the Makefile that it generates”. Again, if the way to do these things is by hand, it’s going to be much more natural to just write it in the main Makefile, and not write a second Makefile and include it in the first.

In conclusion, this method of working is sort of kinda OK if you want to do some simple stuff, but I can’t imagine any serious project with a degree of non-trivial complexity using it, and quite frankly you’re better off doing the simple stuff by hand as well, because EDE is just going to get in your way, and its extra features like making an archive of your code and uploading it to an FTP server are far from being “killer features” that will make the hassle worth it.

EDE with existing code that EDE doesn’t recognize

For any not-officially-supported scenarios EDE has the notion of a “simple project”, which is basically a do it yourself option. ede-simple-project is an EIEIO object that is designed to be subclassed, and you are free to implement all the necessary elisp functions by hand to support whatever situation you have in your project. If you know how to do that, you don’t really need mu help.

Simple projects are defined by writing elisp code in your .emacs file, and are pretty much featureless. None of the major commands like building and creating targets will work, but the more trivial ones like “Edit Project Homepage” or “Browse Project URL” will, as long as you manually set the appropriate variables in your elisp code (since there’s no Project.ede where settings can be persistent, it is not possible to set this via the customization features of EDE). However, like I said at the start of this post, you will still want to use a simple project even if you don’t plan to extend it, just so that other parts of CEDET will have access to the limited metadata it does have. Otherwise, you may have to configure all the other CEDET tools by hand.

I will however show you how to setup a simple C/C++ project, which doesen’t take much knowledge to do. Its EIEIO object ede-cpp-root-project is special because it’s aware of some typical C/C++ idiosyncrasies like the abundance of .h files. A basic configuration can be created like this:

(ede-cpp-root-project "project_name" :file "/path/to/some/top/level/file"
         :include-path '( "/include" "include/configs" )
         :system-include-path '( "/usr/include/c++/3.2.2/" )
         :spp-table '( ("MOOSE" . "")
                       ("CONST" . "const") ) )

Hopefully most of the stuff you see in this code sample is self explanatory even if you don’t know elisp. Essentially what this code does is give the project a name, tell CEDET where the the project root is by pointing it to a file in that root directory, tells it where the include files are (an entry starting with a “/” is relative to the project root, while an entry that doesn’t start with “/” is interpreted as relative to the current buffer), tells it where the system include files are, and then there’s that spp-table variable.

spp-table is where you can list a bunch of C preprocessor defines that are critical for the various #ifdefs in your code. CEDET can theoretically find them while parsing the code in your project, but placing them here explicitly makes sure that they are indeed found when needed.

You can see a full documentation of ede-cpp-root-project in the EDE manual under the Project/ede-cpp-root-project entry. Examining the inheritance tree can give you the full list of elisp “slots” (variables) you can configure in the above template. The full list is this (items you probably not need to set are grayed out):

  • name: String
    Default Value: “Untitled”
    Used for distribution files. No reason to set this since Make Distribution is not supported in simple projects.
  • version: String
    Default Value: “1.0″
    Used for distribution files. No reason to set this since Make Distribution is not supported in simple projects.
  • directory: String
    Directory this project is associated with. Not sure what its use is considering that this can be inferred with the help of the file variable.
  • file: String
    A name of a file in a top level directory of the project.
  • targets: List
    List of top level targets in this project. This is used by EDE when it generates Makefiles, and I’m not sure of whether you have a reason to set this or not in a “simple project”, although I suspect that it might help other tools find some stuff better if you do (I myself do not bother to set it).
  • tool-cache: List
    List of tool cache configurations in this project.  This allows any tool to create, manage, and persist project-specific settings. If I’m not mistaken this section is for plugging in custom tools (notice that the “tools” section of customize-project is always empt).
  • web-site-url: String
    URL to this projects web site.  This is a URL to be sent to a web site for documentation.
  • web-site-directory: String
    A directory where web pages can be found by Emacs.  For remote locations use a path compatible with ange-ftp or EFS. You can also use TRAMP for use with rcp & scp.
  • web-site-file: String
    A file which contains the home page for this project.  This file can be relative to slot `web-site-directory’.  Same guidelines as above.
  • ftp-site: String
    FTP site where this project’s distribution can be found. This FTP site should be in Emacs form, as needed by `ange-ftp’, but can also be of a form used by TRAMP for use with scp, or rcp.
  • ftp-upload-site: String
    FTP Site to upload new distributions to.  Same guidelines as above. If this slot is `nil’, then use `ftp-site’ instead.
  • configurations: List
    Default Value: (“debug” “release”)
    List of available configuration types.  Individual target/project types can form associations between a configuration, and target specific elements such as build variables. Not useful to you unless you plan to write your own elisp functions that use this.
  • configuration-default: String
    Default Value: “debug”
    The default configuration. Not useful to you unless you plan to write your own elisp functions that use this.
  • local-variables
    Default Value: `nil’
    This variable is for setting Emacs variables that you want be set only for this specific project.
  • include-path: List
    Default Value: `(quote (“/include” “../include/”))’
    The default locate function expands filenames within a project.  If a header file (.h, .hh, etc) name is expanded, and the `:locate-fcn’ slot is `nil’, then the include path is checked first, and other directories are ignored.  For very large projects, this optimization can save a lot of time.
    Directory names in the path can be relative to the current buffer’s `default-directory’ (not starting with a /). Directories that are relative to the project’s root should start with a /, such as  “/include”, meaning the directory `include’ off the project root directory.
  • system-include-path: List
    Default Value: `nil’
    The system include path for files in this project.  C files initialized in an ede-cpp-root-project have their semantic system include path set to this value.  If this is `nil’, then the semantic path is not modified.
  • spp-table: List
    Default Value: `nil’
    C Preprocessor macros for your files.  Preprocessor symbols will be used while parsing your files.  These macros might be passed in through the command line compiler, or are critical symbols derived from header files.  Providing header files macro values through this slot improves accuracy and performance.  Use `:spp-files’ to use these files directly. I myself do not bother setting this.
  • spp-files: List
    Default Value: `nil’
    C header file with Preprocessor macros for your files.  The PreProcessor symbols appearing in these files will be used while parsing files in this project.  See `semantic-lex-c-preprocessor-symbol-map’ for more on how this works.
  • header-match-regexp: String
    Default Value: “\\.\\(h\\(h\\|xx\\|pp\\|\\+\\+\\)?\\|H\\)$\\|\\<\\w+$”
    Regexp used to identify C/C++ header files. If you need to set this then wat.
  • locate-fcn: function (or null)
    The locate function can be used in place of “ede-expand-filename” so you can quickly customize your custom target to use specialized local routines instead of the EDE routines.  The function symbol must take two arguments:   NAME – The name of the file to find. DIR – The directory root for this cpp-root project.
    It should return the fully qualified file name passed in from NAME.  If that file does not exist, it should return nil.

EDE with existing code that EDE recognizes

As mentioned before, there are a few use cases for this option: Automake projects, Linux project, and the Emacs project. It is worth noting that in EDE’s eyes, existing Automake project and Linux/Emacs projects are actually two different things. Linux/Emacs projects are essentially “simple projects” that are preconfigured and prepopulated with symbols.
I don’t have much to say about this option. It’s going to be somewhere between “simple project” and a fully featured project type.

Advanced: Low level customization of your project

Most of EDE’s project metadata isn’t accessible from the high level commands described above, so in order to customize it you either have to edit the project definition by hand, or preferably use the customization feature. This feature worksin two possible situations:

  1. Your EDE setup uses a Project.ede file (use case number 1).
    If your setup uses a Project.ede file, the customization feature will take you to a special buffer similar to Emacs’ “customize” buffer.
  2. Your EDE setup does not use a Project.ede file (use cases 2 and 3).
    In this situation EDE is either using Makefiles or some other file to learn about the project’s setup, and the customization feature will take you to these files themselves. For example, if you are in the Linux project, and type C-c . e, it will take you to scripts/ver_linux.

The customization buffers can be accessed via the Project menu, and look something like this:

customizationbuffer

As you can see, the different options are documented. Some of the documentation in there is helpful, other not so much. Since this post is already long enough, I’ll focus on describing what I personally feel is the non-obvious stuff.

  • Default:
    • For Automatic-Dependencies, Include-File, and Inference-Rules you can view this post.
  • Settings:
    • Variables: literally insert variables into the makefile.
    • Configuration-Variables: insert variables that will only be used when compiling with the appropriate project “configuration”.
    • Project Local Variables: set Emacs settings that will be relevant only for this project.
  • Targets:
    • Auxiliary Files under “Local Targets”: when I added header files to my project, they ended up here, which is not what I would’ve expected given the description in the documentation.
    • Linker and Compile options: I didn’t set any, and it appears that this causes EDE to default to GNU tools like GCC.
    • Additional Rules is for inserting hand written rules.
    • LdLibs: despite what the documentation says, this is perfectly supported in Make projects as well.

Confused yet? I don’t blame you.

Conclusion

If you want my opinion: don’t bother working with EDE directly. It’s not really worth it. Just write your own Makefiles, and use EDE’s ede-simple-project or ede-cpp-root-project to define where your includes are, and if you want to- what you web site URL and FTP address is for all those cutesy features. If you don’t want to pollute your .emacs file with project specific definitions, you can do the following:

Place this code into you .emacs file:

;; Create a Project.ede equivalent for ede-simple-project
;; by telling Emacs to load Project.el files
(defun check-for-Project-el ()
  (if (file-exists-p "Project.el")
    (load-file "Project.el")
  )
)
(add-hook 'find-file-hook 'check-for-Project-el)

This function makes Emacs check for the existance of a Project.el file whenever it opens some file, and loads its elisp code. Now, place a Project.el in your project folder, and write all the configuration there.

So that’s it for now. Next time I’m going to cover Semantic.

Posted in Open Source, Software | Tagged , , | 3 Comments

5 reasons that matter for choosing a Linux distribution

A blog post I recently encountered argues that there’s no real significant difference between Linux distributions, and it has a good point. I tried 4 different distros so far (5 if you consider Kubuntu to be different from Ubuntu), and nothing really worked any better or worse. Also, many “significant” differences that do exist either don’t matter, or can be circumvented without too much work. Nevertheless, there kinda still is a difference between Linux distros that can make you favor one over the other. They are not major differences, but they can make a major difference, so I decided to write my own selecting-a-distro guide with a “casual user” in mind (in the Linux world this means that you’d rather not config the shit out of everything, even though you can figure out how to do it). This is what I think you should look at when selecting a distribution:

How bleeding edge is it (AKA what is its release cycle)?

Some distros like Arch Linux roll updates as soon as possible, other distros like Ubuntu have an update “glass ceiling”, but a relatively fast release cycle (new version every 6 months) that pushes this glass ceiling further in an acceptable speed. And then there are snails like Debian, which take years to release a new version, and often fall behind in terms of software upgrades. Can you circumvent this? Sure, you can manually install a PPA for each program, but wouldn’t it be better if you didn’t have to do that?

What is its philosophy about non-free software?

If you’re one of those Free Software hippy nuts advocates who will forego any sort of convenience for the liberty to put your own bugs in the programs you use, then this will greatly matter to you, because you’d want a distro that doesn’t ship non-free software. If you are sane pragmatic, you would probably want a distro that is more lenient about this political issue. While it is possible to get flash and mp3 codecs and proprietary drivers in any Linux distro, some distros keeps these things in their own repository, and some (Ubuntu) even notify you about the existence of proprietary drivers for your system, and point you to the right direction for installing non-free software when needed. Other distros (Fedora for example) refuse to include these things in their repository or help you out in getting them and make you do the work yourself by forcing you to rely on non-official repositories. If you just want to do the things you want to do without having to drop into command line to retrieve missing packages features, you should choose a distro that gives you the smallest number of reasons to do so.

What is its favorite desktop environment?

You can install any desktop environment on any Linux distro, but some distros choose an environment to be their “main” one, and will invest more effort in developing it over the alternatives, if indeed they are going to bother in developing them at all. Examples include Ubuntu, who develop only for Gnome, Linux Mint who release KDE versions some months after Gnome versions, and there are probably more. If you have a favorite environment, you will probably prefer a distro that treats it as first class so that you will get its custom features.

Is it one of those Do-It-Yourself distros?

Most distros are designed to be a ready to use system, and this means making a bunch of choices for the user like what software comes preinstalled and what the default configurations are. There are some distros that take a do it yourself approach, and essentially give the user the task of installing and setting up everything from scratch. These distros include Gentoo, which give the user the option to compile everything themselves, Arch Linux which lets you hand pick all the packages and makes you manage updates on your own via config files, and there’s Slackware, which is not a build-from-scratch distro, but offers almost no hand holding (doesn’t even boot into a GUI). These systems can be a cool way to learn about Linux, but if you just want something that works, you should probably stay away from them.

How good are their software installing tools?

I know how to install software via command line, but most of the time it’s not the most smooth experience. Is the package for Libre Office called libre-office? Or libreoffice? Or livreoffice-3.something? Or something else? I don’t want to guess, and I don’t want to search | grep the repository, I want a big button that says “install Libre Office”. Because fuck making an effort. The problem is, even when distros offer GUI tools for installing software, they are often organized in terms of packages. So far the only distro with a good software manager I encountered was Ubuntu, where the meta packages for installing whole multi-package programs are clearly set apart from the rest of the noise.

So that’s my list. Got any more suggestions?

Posted in Open Source, Software | Tagged | Leave a comment

Metro on Windows 8

So Windows 8 Consumer Preview came out a while ago, and the internet has been abuzz with arguments over whether the new Metro interface that they added works on the desktop. I personally think that the answer is both yes and no, and mostly for completely the opposite reasons than what most people say about it, so here’s my two cents on the matter:

First, people need to calm the hell down. Everywhere I go I see people talking like Metro is about to replace the traditional desktop. I see people complaining that you can’t have multiple windows on the same screen (well, you can have two, but that is of course not much), that they depend on more complex programs than what Metro apps offer, that they have an optimized workflow that is not compatible with Metro, etc etc, and somehow seem to ignore the fact that all of these things are still in Windows 8. None of that is taken away. It’s still all there. Nothing is being replaced. Why is this so hard for some people to grasp? I myself spend something like 99% of my time in the traditional
desktop, and if it weren’t for the slightly redesigned explorer or lack of the start button it would’ve been easy for me to forget this is not Windows 7. Quite frankly a much more logical criticism in my view is “why is Metro even there when it doesn’t do anything for the traditional desktop??”, because that is my biggest reservation about the inclusion of the Metro interface: it doesn’t really adds much to the experience.

Pseudo-power-user-hysteria aside, people are quite right in saying that Metro apps don’t really enhance the desktop. Some of these apps are pretty nice for that “do one task” mindset, and I can’t wait to get Windows 8 on a tablet (or TV), but on the desktop we don’t really need one task apps. Even if all I wanted was to check some email I would still more likely launch Outlook or Live Mail (and quite frankly I wouldn’t even do that, but open Gmail instead), because it’s quite possible that the email I’m about to read will make me want to do some additional task. I don’t want to open a simpler email client just to discover that I need to open a more complex one later. I also don’t need a video player that pauses when I switch to another window. This might seem strange that I want it to continue playing when I’m not watching, but I watch a lot of TV shows on my computer, and some of them have crappy episodes. I don’t want to miss the content, but I get bored watching the bad episodes, so I let it play in the background while I catch up on my  Facebook/email/news. I also don’t need a Metro Internet Explorer, and not just because I use Firefox. These Metro apps are kinda fun to use, and I do open the mail or calendar app from time to time, but overall there’s really nothing about the Metro apps that makes me want to use them over traditional desktop alternatives.

Microsoft did add some integration functionality that has enormous potential, especially with the cloud related stuff. It’s possible that people will find a way to make Metro apps that make sense on the desktop, but these apps will not be tablet apps, and they will be good for different reasons. One app that makes me suspect that is the Pictures app, which is actually not a bad way of viewing your library. Its full screen-ness is good for viewing pictures, it is lightweight and starts quickly, and it aggregates your albums from various sources- hard disk, Facebook, SkyDrive, Flicker. This is great. It actually makes browsing all your galleries much easier. If people could come up with more stuff like that, there might be something good here.

One place where I absolutely love Metro on the desktop is the new start screen, because it is superior in every possible way to the original start menu. Yes I said it. This one change in itself generated so much hysteria that I think I’m losing my faith in humanity. You people liked the start menu?! Really?! Let’s have a short comparison between the menu and the screen, shall we:

The start menu requires much more hand coordination to operate: the targets are small, so they are harder to hit than the large Metro icons. Since they are harder to hit, you need to mouse to them more carefully, which means doing slower movements, which means more muscle tension (when we need to do small, precise movements our muscles tense up). If you have RSI, this hurts. If you have Parkinson’s disease, it’s frustrating. If you value your time, it’s annoying, because it’s slower. Metro on the other hand, works great with big, swooping motions. I saw some people complain about the increased distance that you need to move your mouse, but what they don’t realize is that our hands are built for these kinds of movements, not for small, precise ones, and while excessive mouse  movement can definitely be a problem, you only need to do one such movement on the start screen: you bring it up, use the scroll wheel to get to the right section, and send your
pointer to the icon. On the traditional start menu we first need to move the pointer to the Programs entry, then we need to mouse to the side (another precision movement), then depending on our organization scheme we need to mouse to the app or the folder, and if it’s a folder we need a fourth precision movement. This is crap. The Metro start screen doesn’t require precision movements, nor does it require you to navigate folder hierarchies. You just scroll to the right section and throw the mouse in that direction.

One problem I do see is for track pad users who don’t have a scroll line on the track pad. Another problem someone mentioned is that if you are running Windows 8 in a windowed virtual machine, then the hot corners become extremely difficult to hit. The right way to launch apps is via WinKey + <type>, so I don’t see this as critical problems, but they are problems and need to be addressed.

Overall my view of Windows 8 UI is this: Windows 7 + a new start screen. Metro apps can be safely ignored at this point. Maybe people will figure out how to make Metro apps that make sense on the desktop (the Pictures app definitely shows there might be something there), but so far I don’t see them enhancing the desktop experience in any way. There is absolutely no need for the hysteria though. You can still use it like you used Windows 7, and there’s no reason to give up on its under the hood improvements just because you don’t like features that you don’t have to use.

Update: I thought about this some more, and I realized that the Metro apps also show signs of better integration between the PC and other devices. The X-Box apps kinda scream for a physical X-Box to interact with. The Video app looks like a great TV interface. To me, the need for syncing your devices all the time is the greatest annoyance of the post PC era. You have your Apple-TV like devices, you have desktops and laptops/tablets, you have phones or dedicated MP3 players, portable gaming devices, e-book readers, none of which are mutually exclusive when it comes to content. I can and want to listen to the same songs on my phone and my computers. I want to watch the same movie both on my computer and my tablet. I want to read e books both on my computer and my Kindle. It all depends on where I am and which device I have with me, and the result is that I have to manage this cross-device availability of files myself. The inclusion of the Metro interface definitely seems like it can help with some of that by making Windows 8 computers viable backends for some of that stuff. It can be my Apple-TV-like device because it has the interface for it, and it can be something I’d want to plug my X-Box into and thus have all my X-Box content with me at all times, regardless of whether I’m in front of it or not, because they both plug into my laptop, which I can take with me.

Posted in Microsoft, Software | Tagged , | Leave a comment

Switching to Linux as your main OS? I got good news and bad news for you

Reviewing software is one of my hobbies, and in the course of using Linux (dual booting Ubuntu with my main OS- Windows 7) I got to try quite a lot of the Linux/FOSS ecosystem’s software. Now I feel the need to share my findings, just to give newbies an introduction to what Linux has to offer (and save them some time) and because there are some things I want to get off my chest. The short version is that there are some areas where this ecosystem excels, especially if you are a programmer. These programs show that the open source model can produce some very good results, and that makes the “bad news” I’m going to rant about all the more grim. There are some areas where the Linux experience absolutely sucks, and when you see which categories these areas include, you’ll soon realize that this isn’t going to be “the year of Linux” anytime soon.

In this post I’m going to focus mostly on Gnome programs, because this is what I’m the most familiar with and because I’m in favor of GUI uniformity and generally prefer programs to maintain the overall aesthetic of the OS. As you will see in the bad news section though sometimes using a program with an alien look and feel is going to be the only way to maintain some sanity in the Linux world. I’ll mention some KDE apps whenever I can too. One thing I’m not going to do is address the fact that you can load Windows programs using Wine or other emulation software. When you bring emulation to the table the whole discussion about software “ecosystems” becomes moot.

Good news

So let’s start with what the Linux ecosystem can be good for:

Browsing the Web

You can get Firefox and Google Chrome on Linux. What else could you possibly want? Firefox will even make itself look native to Gnome, and Google Chrome doesn’t look too out of place either. I suppose you can also install Opera, if you really want to. There’s definitely no compromising here.

email (and calendar)

For email the two most prominent options seem to be Mozilla’s Thunderbird and Evolution. Thunderbird is a perfectly fine tool for doing email, and though it doesn’t come with a calendar/TODOs/Tasks you can install the Lightning addon and get that functionality as well. It’s definitely not as feature full as Microsoft’s Outlook, but it should be good enough for most people, especially if you utilize the active addon community. Mozilla like to support each OS’s aesthetic, so Thunderbird will make itself look native to Gnome, and in fact does a much, much better job at it than Evolution, Gnome’s official default email program.

If you want the Outlook experience, there’s Evolution. It does pretty well on the feature checklist but it feels clunky and is quite ugly. It used to be pretty buggy as well, and I almost wrote this paragraph with the goal of ridiculing it, but I gave it another try and it seems much more stable now. Going to Evolution after using Outlook 2010 feels like a huge step backwards in terms of user experience, but at the very least it is a capable program.

Evolution
Evolution at its fugly state. Notice the inconsistent spacing between the top icons, inconsistent vertical padding in the line where Show and Search is, overuse of the color grey, and terrible typography.
 

To be honest I wasn’t sure whether to stick this in the good news or the bad news section because I’m basically saying that out of the only two options worth using one of them involves a huge UX compromise, but the truth is that it doesn’t matter. So much email work is done inside the browser nowadays that even if neither of these programs works out for you, using Gmail or Hotmail (yes I said Hotmail) is a perfectly good experience.

Watching movies

If you just want to watch some movies, the Linux world has got you covered too (once you manually download codecs that normally come bundled with other OSes that is). VLC is available on Linux, and that right there should be enough for everybody. VLC is an outstanding, robust player, and often succeeds in playing a movie where other players fail. But some of the programs that come (or used to come) bundled with Ubuntu are perfectly fine too. These include Totem and MPlayer. Though I admit I did not give them a thorough testing, they both seem perfectly fine for just playing movies.

Generally controlling your computer

The Linux Terminal is pretty magical. This is definitely one of the best things in Linux. There’s seemingly no limit to what you can do with it thanks to the easy, fluid way in which it allows you to invoke programs and let them interconnect.

monkeybrains-30mbps-internet-connection-speed

Want to install a program? Typing

sudo apt-get install <name of program>

will automatically locate, download, resolve dependencies, and install the program. Have a source tree with hundreds of files and you need to find where a function is defined? Typing

grep –rn <name of function> *

will search all the files and display in which files and in what line does the function name appears. Need help with learning some command? Type

man <command name>

and you’ll get the command’s manual page. Want to cure cancer? Type

sudo –xv cure-cancer *

and it’ll cure cancer (this feature is in pre alpha and will most likely not be available in your distribution).

You can write scripts that can do pretty much anything (for example, an alarm clock that launches a music player with a playlist and gradually raises the volume), and there are many programs in Linux with great command line interfaces (many are even designed as command line programs first), including the device driver system (yes, you can interface with drivers via command line). In short, there’s a reason why so many Linux screen shots have the terminal in them, and while there’s a somewhat steep learning curve to it, it’s well worth it.

Eye candy

Both KDE and Gnome made a (not so successful) effort to make themselves modern desktop environments. While these attempts were met with much criticism (Gnome 3’s usability was criticized by many, and KDE 4 had a rocky start too, and in fact made long time user Linus Torvalds, as well as many others, switch to Gnome), but they did do a not too shabby job at making Linux look nice (although quite frankly Gnome could be much better). But of course the real star here is Compiz, the kick ass program that brings all sorts of cool stuff into Linux desktop, including many useful tricks and customizations. As the following video demonstrates, it can make Linux look outstanding.

Downloading stuff

Linux is pretty good with all sorts of downloads. Need to download an entire source tree? Most source control systems can be invoked via the terminal, all you need is a link. Need to work with FTP? There’s FireFTP for Firefox or the standalone FileZilla. Bittorrent? Transmission, which comes with Ubuntu is kinda shitty, but Deluge is pretty much a clone of the official Bittorrent client, which is good.

ebooks

There isn’t really much choice for ebook readers on Linux for the Gnome environment, but KDE has one of the best programs for ebooks I ever encountered.  Calibre is not only an ebook reader, but also an ebook library manager, and can do pretty much anything you may want to do with your collection of ebooks: edit metadata, convert to other formats, sync to various ereader devices, buy books, and even fetch news from web sites and convert them to an ebook. What is also great about Calibre is that its creator appears to have a clue when it comes to user interface (something I can’t say about more than half of FOSS projects out there). It’s a little messy and verbose, but aside from that it gets a lot of things right, and it doesn’t look too out of place in Gnome either.

Text editing

The *nix community dates back to the days of terminals and computers without mice, so text editing is a problem that was optimized the hell out of many years ago, and the options are aplenty.

Most of the hardcore Linux programmers either use Vim (or Gvim), or Emacs. Both are keyboard centric editors that have a huge collection of time saving keyboard commands for moving through and manipulating text, not to mention a variety of plugins that can do anything from syntax highlight and auto completion to bringing an email client and file system browser right into the editor. They both have a very steep learning curve, and will take you months to master, but people who manage to get through it usually say it was well worth it for their productivity. The following videos demonstrate some of the basic capabilities of these editors, as well as some examples of plugins.

A demonstration of basic Vim editing. Notice the cut ‘n’ paste action, the quick jumping around, and split screen/command line.
Here’s a demo of some of Emacs plugins

Emacs is extremely customizable. Almost every aspect of its behavior can be changed by the user, including the smallest of nuances like what punctuation marks define the end of a sentence. In fact, it’s so customizable that it has its own programming language (elisp), and can run internally programs written for it, which include games, a calendar, a news reader, and more. You can use it to read email, browse and make changes to your file system, edit ASCII art, and lots of other stuff. Yes, ‘holy crap’ is an appropriate reaction.

Emacs is effectively an operating system, which is how Vim fans like to make fun of it, describing it as “a great operating system, lacking only a decent editor”. This might be an overkill for a text editor, but it does come in handy. You can make it act in any way you want to (it is not uncommon for people’s .emacs file, the file Emacs reads to load user customizations on startup, to have hundreds of lines), and with addons for source control integration, compiler/debugger integration, and auto completion, you got yourself a nice IDE.

Of course, Vim has quite a lot of plugins as well, including the above mentioned source control, auto completion, compiler/debugger integrations, and so on, just not the superfluous features and customizations. Some people even argue that once mastered it’s better at actual text editing, and it doesn’t have the reputation of giving you RSI. I’m doing it a disservice now by talking so little about it (it really is a great editor), but I currently use Emacs and therefore am not very familiar with Vim.

If you don’t program or don’t feel like learning how to use these complicated text editors there’s also a pretty good basic editor called gedit. Its keyboard shortcuts are very modern (Ctrl-C for copy, Ctrl-Z for undo, and so on. Emacs and Vim’s keyboard commands are nothing like what people are used to nowadays), and it has some plugins that give it the basic functionalities people expect from a text editor (syntax highlight for example. I absolutely love the Oblivion theme). It’s not a replacement to Notepad++ by any means, but it has the potential to get there given enough time. For really quick edits there’s Pico and Nano, which are keyboard based but very simple editors. At any rate, regardless of how hardcore you want your text editing to be, the Linux ecosystem has the right tool for you.

Programming

Despite what some Linux fans might say to you, or the efforts of companies like Canonical, Linux is still an OS “by programmers, for programmers”. This is bad for the average user, but pretty good for programmers themselves. Programming on Linux is a pretty good experience, and while I maintain that being a programmer in the Windows world is better, the open source community definitely gives them a run for their money.

I already mentioned Emacs and Vim as great editors/IDEs, but it doesn’t end here. If you want a true, full blown IDE, you have Eclipse. It’s outstanding for Java, and pretty good for C/C++. For C# there’s MonoDevelope, which is also pretty good. You can get all kinds of source control programs on Linux, and most importantly- git, Linus’ second best achievement after Linux. The Linux terminal and the outstanding scripting capabilities can be very useful, and living in the open source world means that source code is always available. Many tools have good interoperability too, so it is not uncommon to discover that your favorite debugger tools are usable in other environments as well.

I’m sure more experienced Linux developers than me can give more examples. Many of these things are available on Windows as well, but some of them (specifically the scripting and anything terminal related) feel more native in Linux, and there’s definitely a certain plus in how heavily text oriented pretty much anything in Linux is.

The bad news

Office

I wasn’t sure where to stick this section because Libre Office- which now comes in Ubuntu by default- is a capable office suite. You can actually use it and be satisfied feature wise. The problem is that compared to Microsoft’s Office it’s a huge mass of suckiness (I can already hear the Ribbon haters massing with torches and pitchforks behind my door). Libre Office started as Open Office and was meant to be a clone of Microsoft Office 2003. Unfortunately, Office 2003 had a lot of UI defects: it felt overly complicated, it was hard to find stuff, it was hard to get to features, and generally the UI was a mess that didn’t help anyone. …and they cloned these defects as well. I know that there are people who whine about Office’s Ribbon UI (still. It’s been 5 years, shouldn’t they have gotten used to it by now?! I call PEBKAC), but surely even they can’t defend the chaos of crap that is the Libre Office GUI. You open one of the programs and are presented with almost 50 small, unlabeled, mostly ambiguous buttons and toolbars without proper organization. After you’re done scanning all of them and discover that your feature isn’t there, you then need to go to the menus and scan over 200 menu items. I think I read somewhere that this was how they tortured political prisoners in North Korea. They sat them in front of Office 2003 and told them to search for features.

The first minute summarizes the Office 2003 user experience.

The problem with those Office clones becomes apparent the second you open a document generated by another office suite and see it all messed up. The margins squeeze everything in, formatting gets broken, the fonts change, pictures move on top of other things, the punctuation in RTL languages gets all screwy (and god help you if you happened to intermix RTL with LTR languages, you’ll be lucky to understand half the sentence). What a great experience! But hey, at least it has 7000 buttons on the toolbars for one click access to all of your features, isn’t abundance of buttons the definition of good UI and UX? Open Office isn’t any better, and in fact is slowly dying. There’s KOffice on KDE, but I don’t believe it’s much better. Your best option might well be Google Docs, which isn’t nearly as fully featured, but is at least good enough for simple scenarios. You can try Microsoft’s web apps for Office, but they are frankly worse than Google Docs in terms of features. In short, you can’t win.

Music and general media library management

There are actually many programs for managing music and video media on Linux: Exaile, Sonata, a-tunes, Clementine, Amarok, Rhythmbox, Banshee, and probably many more. Most of them suck hard. Trust me, don’t even bother downloading them. It’s not worth your time. From atrocious interfaces to lacking features, if there’s a deal breaking defect for a media library player- they got it, and in multiple quantities. The only player that is kinda decent is Banshee, and that’s a sad state of affairs since it is nowhere near the quality of Zune, iTunes, or Media Player. It *is* a capable player: it has iPod synching, integration with online stores/Wikipedia/Last.fm/YouTube, the richest selection of media type management (music, video, audiobooks, and podcasts), and even album tiles in the music browser section (you can pretty much forget about iTunes-like “grid” navigation, NONE of the players have such a thing implemented, but Bunshee at least comes somewhat closer to it).

The problem is, as always, the user experience. Banshee’s UI is kinda haphazard in that “let’s solve UI problems by introducing a new rectangle” sort of way, and is frankly pretty ugly (LOL oxymoron). It’s not feature complete either, and some parts of it feel half baked; for example it doesn’t really manage audiobooks or videos, since you can only browse your collection as one huge list, so while you’ll probably discover that it’s OK as a music manager, it leaves a lot to be desired for everything else. Then there are the random annoyances like how just selecting a track, without double clicking it, interferes with playing order (next track is the selected track, not the next track in the album), and a bunch of Linux pathologies like stuff not working (this one is more individual of course). Overall, while it technically gets the job done for managing music, it still feels like a step back, both in terms of the design, and the missing features, and the pointless extra features (yes it’s very nice that it can integrate all kinds of web services, but how about giving me a feature that is not better served by a web browser?).
Rhythmbox is sort of OK too, but it’s effectively a simpler version of Banshee, so there’s no point in choosing it over Banshee.

comparison1
Windows Media Player 12 vs. Banshee. Click to enlarge.
 

There is another player out there, that even fixes some of Banshee’s defects, called Nightingale. Nightingale is Songbird for Linux reborn. After Songbird dropped Linux support, some people decided to fork it and continue to make a Linux version. It looks great, in part because it uses its own graphics engine, not relying on Gnome’s GUI, and doesn’t even look too out of place in that environment. It’s also a capable player, and the most iTunes-ish program you may find for Linux at the moment (there used to be a Grid-view addon on Songbird for example. It was buggy as hell, but it was there). On the down side it’s still in development, and development is going slow. There are currently no addons, no webstore support, and it keeps carrying that integrated web browser around, relying too much on its utility if you ask me. Maybe someday it will mature, but for now there’s no media library player/organizer that I’m satisfied with.

Photo library management

If you thought the situation with music library managers is bad, this category is even worse. All the programs suck, and then there aren’t even that many of them. For starters pretty much all of them don’t actually manage your collection. When you have a collection of photos you want to be able to find photos of a certain person, or a certain event, or maybe even location. You will be lucky to find a half assed support for at least one of them in a photo manager on Linux. Most of them don’t have face detection, actual geolocation data, or the ability to join photos into albums. You usually just get plain tags and sort by date, which is not even close to what you need: sorting by date fragments multi date events like a vacation, and substituting people, location, and event tags with plain tags will lead to a tag explosion problem.

None of them help you tag photos, and many don’t even bother to create a proper GUI for it- you have to use menus. Tagging a large collection is a prohibitive chore even with help from the software such as face detection and reading geolocation info, so when you not only don’t provide assistance but actually make it RSI inducing it just becomes unacceptable. Then there are design defects, the let’s-throw-some-buttons-in-there/GUI-is-just-a-map-from-function-to-button UI design, and the typical OSS pathologies like stuff not working. Showtell is the only app with some promise (in the sense that what it does have doesn’t come with 1001 things you want to fix), but it doesn’t really do anything well beside viewing the pictures. It’s pretty low on features.

When it comes to managing your pictures, your only choice is to use Picasa, which has a “Linux version” that is really a Windows version run through Wine. It is behind the Windows version in terms of features, and it doesn’t look native at all, but you will not find an app in this category that does more things right than Picasa.

Games

You can pretty much forget about being a gamer on Linux, unless you don’t mind to wait years upon years until Id Software open sources their latest now-obsolete-by-time 3D shooter. Oh and there’s also a bunch of open source games, which all suck, especially those always-8-years-behind-the-mainstream-in-terms-of-graphics open source shooters. Aside from the very rare gems that find their way to Linux (like Machinarium), mainly because they were developed on a platform agnostic technology like Flash, you have nothing to look forward to.

Not sure

Advanced creative work

I’m a software engineer, not audio editor, animator, 3D modeler, or designer, so I really can’t tell you if any of these needs are properly met. I can tell you that if you need to create UML diagrams you’re out of luck. I couldn’t find a program that even came close to Microsoft’s Visio, and what I did find was barely usable. If you need a Photoshop replacement you’re screwed as well. The biggest contender seems to be Gimp, and it has one of the most terrible user interfaces I’ve ever seen. It’s packed with features that are useful and yet I can’t bring myself to use it for more than 5 minutes.

6a00d8345282f769e200e54f3187578834-800wiThe GIMP user interface.
 

I don’t know much about about audio editing but Audacity seems to be fine for basic audio work. There are 3D modeling/animation programs that certainly look impressive, although looks are sometimes misleading (for example Blender looks impressive but I heard from people who know about this stuff that it has a very non-standard user interface and you aren’t doing your career any favors by learning it). Overall, you’re mostly on your own here.

Conclusion

So as you can see, moving to the Linux world will involve some sacrifices, unless you’re willing to use Wine to run Windows programs. If you ask me, my ultimate Linux setup is: Firefox (with Speed Dial, MultiLinks, Customize Your Web, AdBlock, and Flash Block), Thunderbird with Lightning, VLC or MPlayer, Deluge, gEdit for simple text editing, a combination of Emacs and the terminal when I’m doing systems programming at work, Eclipse for hardcore projects and Java in general, git for source control, Banshee for music, and Picasa. I don’t edit graphics on Linux if I can help it, and I prefer to avoid using Libre Office whenever possible.

That’s all. Hope I was helpful.

Posted in Open Source, Software | Tagged , | 2 Comments

Software engineering in the Technion – semester 5

This is the fifth post in my series about studying software engineering in the Technion. If this is your first time reading a post in this series I recommend that you read the introduction post first (go ahead, it’s rather short). If not, I’ll give you a short reminder of where we are:

The fifth semester is in the “second tier” of the degree, which is where we concentrate on the essential computer subjects of the degree.

All the courses in this post are mandatory (including Biology 1, in the sense that we are required to study 8 academic points of science courses. The choice of Biology 1 as my final scientific course however was mine). 

Table of contents:
  • Parallel and Distributed Programming 
  • Theory of Compilation
  • Computability Theory
  • Introduction To Software Verification
  • Numerical Analysis 1
  • Methods In Software Engineering
  • Biology 1

The fifth semester

frame

Parallel and Distributed Programming
3 academic points. Computer science faculty course.
Prerequisites: Operating Systems.

What is it about: The challenges of programming systems with multiple processors.

About the course:  Parallel programming (when there are multiple processors working on the same task simultaneously) is the new frontier, because the method of improving performance by literally making hardware faster is not practical anymore. Today, if we want to continue getting better performance out of computers, we have to do it by utilizing multiple processors, and that brings with it a bunch of tricky challenges: we want to distribute the workload evenly to get the maximum use out of our multiple processors, but we can’t always know in advance how to distribute the work in such a way. We can try to distribute the work as it happens, but now we have a new problem: managing the whole thing, which can lead to a lot of costly communication between the threads. We also have to deal with memory issues: in some systems memory is divided between the processors; accessing your own memory is fast, but accessing the memory of other processors is slow. How do we distribute the workload in such a way so as to minimize how often we need to get data from someone else? And then again, in some systems you can’t access somebody else’s memory, so in order for them to share information we need to set up communication between the processors, and this is a problem in itself (how to do this safely, how to do this sparingly).

parallelProblem
A simple division of work between multiple processors.
 

This course teaches how to deal with these things, and covers, along with the theory, a bunch of libraries and APIs that help us handle the challenge of parallel programming. I was pleasantly surprised to learn that there are some very useful libraries for concurrent programming. Up to this point I was used to doing everything manually: create threads, use shared data, synchronize it manually, etc etc. It can be very tricky work, and the libraries they showed us in this course did help with some of the challenges.

Of course, the incredibly low difficulty of the assignments might’ve contributed to my favorable impression of these libraries. I think that overall the lecturers in the computer science faculty are perfectly fine (some even good), but something went really wrong with this particular course. The lecturer was apparently crappy (I wouldn’t know because I didn’t go to most of the lectures, but I heard more complaints than usual), the assignments were ridiculously easy to the point where they didn’t contribute to our understanding of the material, it took forever for the staff to check the homework (assignment 2 took 10 weeks!!), the TA in charge of most of the assignments was extremely unhelpful and you could barely get an answer out of him, there was confusion about what material is going to be in the exam, and as if that wasn’t enough they gave us an unsolvable question in the exam. There was a lot of outcry about this, and a representative of the student union put a lot of pressure on the lecturers to remedy this, so in the end we got a very forgiving exam check (one multiple choice answer had 15 possible answer combinations that got full mark!), and we got to enjoy the ever so rare square root factor (a “factor” is a correction to the grade that is introduced to bring the course GPA to a certain level. Usually a factor is something like +5 points, or +10%. A square root factor is grade = 10 * sqrt(grade). I don’t know the mathematical rationale behind it, but it’s one of the strongest factors you can get). But in the end of the day, this was just treating the symptoms. On the previous semester 60% failed the exam. They got a factor of final_grade=exam_grade*100/86+5, and as you can see, the crappiness repeated itself on my semester.

What is it good for: Parallel programming is extremely important and can be found in many programming problems, from networking to GUI to operating systems, to many other things (this course focuses a bit more on the dealing-with-hardware-related-challenges than the purely software related problems though).

compilationTheory of Compilation
3 academic points. Computer science faculty course.
Prerequisites: Automata and Formal Languages.


What is it about: How to write compilers. What’s a compiler you ask? I’ll tell you. This is how programs are written nowadays: programmers write the program in some programming language like C/C++/C#/Java/Pascal/whatever, which is easy for humans to read and understand but means nothing to the computer, and then a program called a “compiler” is activated and translates it into computer language, which is hard for us humans to understand (thus the reason why nobody uses it to write programs) but the computer understands perfectly.

About the course: I’m not really into compilers so I couldn’t get too excited about this course, but objectively I can tell that it has nice things about it. For example, you get to write a really simple compiler in one of the homework assignments, which is cool. Another interesting part was the general structure of compilers; turns out that compilers are really modular things, and there are even various tools that can generate some parts of compilers automatically: you give them a language definition and they generate the appropriate parser. This is possible thanks to the theory you studied in the course Automata And Formal Languages, which shows that this seemingly archaic theory actually has really concrete applications.

compilerThe stages of compilation.
 

But most of all I enjoyed the part where the lectures ended a full month ahead of time. Turns out that this course is relatively short on material, which was a refreshing change. It’s not often that you get some kind of reprise in the Technion.

What is it good for: Well, if you want to work on compilers, this is definitely the course to take. But it is also a pretty good demonstration of how to do parsers and the various tools that are available for the tasks. Like our TA said in one of the tutorials: sometimes you may get a file with raw data, and you will have to do a lot of work to make sense of it. The kinds of tools you learn in order to write compilers are also very useful for automating these kinds of tedious tasks (if you know how to use them).

computabilityComputability Theory
3 academic points. Computer science faculty course.
Prerequisites: Algorithms 1.


 
What is it about: This course covers two subjects: computability theory and complexity theory. Computability theory asks the question: which problems can be solved using a computer? Complexity theory asks: given a problem that can be solved using a computer, how much resources (time and space) do we need to solve it?

About the course: This is yet another course that is very logic-like, and it also happens to be my favorite logic-like course, because the theory here is really clever and elegant, with some pretty cool conclusions. Just the way it describes computations is quite remarkable:

Imagine that you have an ABC that consists of only two letters- 1 and 0, and you have a machine with a tape on which lots of 1s and 0s are written. This machine can read the tape and selectively change the “letters” based on some internal mechanism. (This machine is called a “Turing Machine” after Alan Turing who invented it. )

Sounds too simple to do anything useful? Well, apparently this deceptively simple mechanism is expressive enough to describe all computations. Just think about this for a moment. This seemingly dumb, limited machine can describe state of the art artificial intelligence. That’s quite amazing to me.

turingMachine
An artist’s representation of a Turing Machine.
 

In the first half of the course you learn that there are problems that can’t be solved (the halting problem), which has all sorts of interesting repercussions like the fact that it’s impossible to write a program that catches all bugs in another program. You learn that some problems are intrinsically harder than others, and that if you can figure out how to solve them you will be able to solve a whole array of other problems, which is an idea that continues in the second half of the course as you learn about complexity. You even get to see a mathematical proof that it is possible to write computer viruses (it’s possible to write a program that copies itself), or that operating systems are possible (you can write a program that runs other programs).

In the second half you learn that some problems require more resources in terms of time and space than others, and that there is a hierarchy of difficulty in here as well. You also learn the meaning of that iconic question in computer science: does P=NP, which in other (very simplified) words basically asks “given a problem, is finding the solution as difficult as validating the correctness of a given solution?”. If the answer is yes, then the whole field of cryptology will collapse (or at least a big part of it) because many methods of encryption rely on the assumption that the answer to this question is “no”. For what it’s worth, most experts believe the answer is indeed “no”, in part because it flies against centuries of experience in the field of problem solving. Scott Aaronson of MIT put it rather eloquently when he said:

“If P = NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in ‘creative leaps,’ no fundamental gap between solving a problem and recognizing the solution once it’s found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss…”

Scott Aaronson, MIT

This course had a few interesting idiosyncrasies. First one was that the exam had two parts, and you can attend which ever part you want as long as you do both parts by the end of the second terms. That’s a very welcomed change because it definitely reduces the amount of work you need to invest into the overall exam (since you can split it). A less welcome change was the fact that you didn’t get any grade for the homework. Instead, you could come to a “workshop” (AKA the TA’s office hours) and ask questions about your solution. While the personal attention was apparently much appreciated by the students, I really feel like this no grades thing was terrible, because no matter how much self discipline you have, it’s just too easy to postpone the homework indefinitely when you have pressure from other assignments for which you do get a grade. I consider myself to be pretty self disciplined (in fact, I often feel disapproving of how much less disciplined others seem to be), and yet even I ended up postponing the assignments until- well, until the exam times. The result was an almost perfect score on the first half, and a barely passing grade on the second half.

a_bunch_of_rocks

What is it good for: For a software engineer- not much. This is really a course for computer scientists. But one practical thing you will get from it is the ability to identify NP-Complete problems, which will help you to avoid trying to solve them (though it haven’t been conclusively proven they can’t be solved efficiently- that’s the P=NP question- no one managed to solve them efficiently yet, so you should probably give up and come up with an approximation solution).

whatIntroduction To Software Verification
3 academic points. Computer science faculty course.
Prerequisites: Logic and Set Theory For Computer Science.


What is it about: The theory behind methods for automatically verifying the correctness of programs.

About the course: This course is pure crap. Well OK, it’s not pure crap, but it’s pure crap that it’s in the mandatory course list. The stuff they teach you is out there, and very occasionally it’s even used in real projects, but most people will not meet it in their careers, and as such this really should’ve been an elective course.

The course is very logic like, but despite my general liking of logic I did not enjoy this one at all. It had some very annoying homework (akin to washing floors with a tooth brush), strange syntax and algorithms, and for some reason they decided not to have lecture slides like every normal course, and instead taught by writing on the board like it’s the middle ages. Copy stuff by hand?! Show up for lectures?! Fuck that shit.

The general gist of the course is this: it’s impossible to verify correctness in all cases, so we have a bunch of different methods for different situations. The first half of the course focuses on proving correctness (Floyd and Hoare methods), and the other half focuses on proving the model (temporal logic, CTL, BDD, and some other crap).

What is it good for: Only for working with automatic verification.

whatNumerical Analysis 1
3 academic points. Computer science faculty course.
Prerequisites: Introduction To Computer Science M, Algebra A, Calculus 2T.


What is it about: Something to do with doing advanced math with the aid of computers.

About the course: I didn’t take this course yet. Come back in 6 months.

What is it good for: —

softwarearchitectureMethods In Software Engineering
3 academic points. Computer science faculty course.
Prerequisites: Data Structures 1, Object Oriented Programming.


What is it about
:  The development cycle: its various stages (requirements definition and analysis, design, testing, and development), how to do them right, and how they are connected.

About the course: This is a pretty important course to take, because it’s an introduction to how software projects are managed in real life, and what kinds of challenges a software project faces (basically, it’s taught from the project management point of view, rather than the programming-task-specific view most courses take). It discusses the software life cycle, from design to development to testing to support, gives an introduction to UML (a graphical language for design. This takes the greater part of the course), covers various project management methodologies (extreme programming, scrum, waterfall model, rapid prototyping, etc), and discusses testing.

diagram02
Scrum is one example of agile development.
 

The homework assignments are used to simulate all the development stages that are taught in class: students are divided into groups of 5, and are given a “client story”- a description of a problem or a product that some fictional client needs solving/doing (in our cases it was something called  “GPSWiki: Motorists’ Joint Road Knowledge Base”, which was supposed to allow motorists to track their rides and submit them online), and the students are supposed to do the requirements and “use case” analysis, software architecture and deployment model, come up with a test plan and a work plan, and implement a prototype.

275px-Windows_2000_architecture.svg
The Windows 2000 architecture.

It was pretty cool in theory, but my experience of the process was soured by two members going AWOL on us, which left us unable to implement all the requirements on time. We the remaining three ended up sending an email to the TA in charge and got our grade “fixed” by a factor of 1.5, so I still got a descent grade for it, but it was after a lot of annoyance and resentment on our part. It was especially annoying to me because I was the project leader, and I was the one who had to deal with these two. On the following semester I- like most of the students of this course- took the first phase of the Yearly Project In Software Engineering, and the same thing happened in one of the groups that got one of the deserters. That group got broken up in the middle of the semester and their members were scattered between the other groups. The guy who arrived to our group told us that the deserter was given a task that could be done in 20 minutes, delayed it until the very end, and then failed to do it.

The moral of this story is: choose your partners well.

1415714204_cbd03aba23
Of course, real projects don’t always go according to plan either.


What is it good for
: software engineering in general.

biology1Biology 1
3 academic points. Biology faculty course.
Prerequisites: None.


What is it about
: This is the entry level course in biology. It focuses greatly on the anatomy of the cell.

About the course: I took this course as my “third scientific course” per the requirement to study 3 scientific courses, and it was definitely a refreshing change of pace. It involved a great deal of memorization, which is rare in my degree, and seemed to be from a completely different world to me. I literally felt like different parts of my brain were at work here.

It was pretty interesting; there was a lot of chemistry involved, which was surprising to me, but that too was kinda cool. I guess the thing that caught my interest the most was how incredibly complex cells are. There’s a whole array of various chemical mechanisms and organs (yes, cells have internal organs) that allow the cells to do very complex things: metabolize materials, expel waste, be selective in what they let in, divide (which is a much more amazing feat once you know that cells have internal organs), and more. The chemical parts had their own surprises as well; turns out the various materials have a very Lego like quality in the way they mix and interact, and some of the products of those interactions can take on functional properties beyond just being a mass of stuff. That’s how the original cells first originated.

 

eukaryotic cell
An Eukaryotic cell.


This course is notable for being a web oriented course. All the lectures are recorded and provided on the course site, so you don’t have to go to lectures at all and are in fact encouraged to study using the recorded lectures. Each lecture has a forum, and a selection of questions. There were also more formal homework assignments but you didn’t get a grade for them, and instead the solutions were published later on for you to test yourself. Apparently this whole thing is part of a pilot, testing the idea of making this course entirely web based (replacing formal lectures). Not sure what the decision will be, but it sure felt quite possible to me.

What is it good for: Well, not much for a software engineer, though it’s important for someone who wants to study bioinformatics (a combination of biology and computer science).


So that’s the end of the fifth post. Hope you liked it. To find the rest of the series:

Previous entry Index Next entry

Posted in Academia | Tagged | Leave a comment

Software engineering in the Technion – series introduction

So, you decided to study software engineering in the Technion.

PipBoy_T_Shirt_Design_by_TheCoconutG[1]

You will get to be one of Israel’s best and brightest.
Technion students get laid more often, earn tons of money, and kick ass at rocking being awesome. …and while the computer science faculty was ranked “only” 15th in the world by the Shanghai university ARWU, at least none of our graduates ever caused a resonance cascade scenario that brought down an age of slavery by otherworldly aliens on the entire planet.

But surely you have some questions lingering in your mind about what you will study and what the experience is going to be like, even if you already decided to study it based on some prior knowledge/experience. Alternatively, you may be a layman considering going into this field that you know nothing about, or you may be just curious about what software engineering is all about. This is where this series of posts comes in. In it I’m going to describe the curriculum of a B.Sc. in software engineering as it is studied in the Technion, and is going to be described in the same order as is suggested by the institute (you can do the courses in any order that is convenient to you provided that you always have the necessary prerequisites, and many people, including myself, deviate from the suggested order in practice). Where personal choice is given to the student (mostly in the later stages of the degree) I will describe the courses that I personally chose to do.

sw_thumb11

The B.Sc. degree takes 4 years to complete, and is divided into three tiers; the first tier consists of the first three semesters and is all about teaching you the fundamental mathematical theories and basics of computer science (hardware and software). Frankly, this part of the degree sometimes feels like you are doing a math degree and not a software engineering one, especially in the first year. These non-computery subjects tend to either outnumber or equally match the computery ones, and while there’s a reason for that it can be a bit bumming for some. The first year is also when you are considered to be in the general 4 year computer science program. You get to switch to the software engineering program once you accumulated the equivalent number of academic points of 2 semesters.

Then comes the second tier, which consists of semesters 4 and 5. It is then that you are taught the essentials of your profession (of course, the first tier has some computer stuff too, but now it’s all about computer stuff). The third tier is where you get to choose what to specialize in. You have about 20 academic points to spend on whatever tickles your fancy, and ideally they should all be spent on the 5 “core” courses from which you are required to choose only 2, but you can of course branch out if you please.

The official requirements are as follows:

  • 160 academic points, of which:
    • 118.5 are mandatory courses academic points.
    • 31.5 are faculty personal choice points, of which:
      • 6-6.5 points should be courses from the “core” list (Project Planning and Management, Computer Security, Database Systems, Introduction To Artificial Intelligence, Software Design)
      • At least 15 academic points must be computer science faculty courses
        • this must include at least one project (beyond the mandatory “Yearly Project in Software Engineering” course).
      • The rest of the academic points can be from either list A (computer science faculty courses), or list B (the list of all the other courses that appear in the different programs in the faculty. This includes courses from electrical engineering faculty, industrial engineering and management faculty, math faculty, and various sciences).
    • 10 free choice academic points. Most people spend them on humanities courses.
  • In the mandatory courses, you are required to take:
    • 2 sports courses
    • A total of 8 academic points in scientific courses. One of them must be Physics 1M, while the others can be chosen by the students, provided that in doing so they complete one of the following “groups” of courses: Physics 2MM OR Biology 1 + General Genetics OR Fundamentals of Chemistry + Organic Chemistry OR Fundamentals of Chemistry + Physical Chemistry.
    • If your score in the English section of the psychometric tests (the Israeli version of the SATs) is below a certain grade, you are required to take Technical English. Otherwise, you are exempt (you get the 3 academic points without the grade).
  • Preparatory courses: If you did not study physics in high school you will be required to study the high school version of Physics 1M before you can take this course. Same is true for Physics 2MM if you choose to take it.

Happy reading:

Posted in Academia | Tagged | Leave a comment

Best of Microsoft Research YouTube channel

The reason I like Microsoft better than their “direct competitors” (Google, Apple) is that they have a good balance between making products and creating new technologies. Google has awesome computer scientists and a lot of really good ideas, but they are not very good at turning these ideas into products; they have some successful products (Gmail, Android, Google Earth), but also some spectacular fails (Buzz, Wave, Orkut) or products that aren’t going anywhere (Reader), and up until recently they had some of the most mind-numbingly boring designs ever (thank god that’s over). Apple is awesome at user experience and software design, but they are really boring as a tech company; ever since their pioneering of the digital music business and the invention of the iPhone everything they did was pretty much just taking existing ideas and making a really good version of them.

Microsoft however is like the fusion of the best traits of both companies; despite what their reputation may suggest, their good (even great in some cases) products still outnumber their failed ones, and at the same time they have a huge R&D budget that yields a lot of interesting stuff. Which brings me to this post:

Today I want to highlight some of the cool stuff they have in their YouTube research channel (some of which already made it to actual products), and the following is a list of my favorite videos. Before we get to it though I’d like to clarify something first because I see people not “getting the point” quite a lot on the internet. The point of a research center is not to produce products, it’s to test ideas. None of the research that goes on in Microsoft’s research centers is tied to specific products or meant to have a business idea behind it, and that’s how a research center should be. These ideas are studied and prototyped, and only later integrated by engineers into specific products, often not in the specific way you may have seen them in a demonstration. So please don’t leave comments saying stuff like “when will we see this product” or “I would never want to use it like this”. The LightSpace project is a great example. Would you want to put something like this in your house? No. But that’s not the point. The point is “let’s see what we can do with depth sensing”. Figuring out “what is it good for” comes later (maybe integrating it into research about holograms, maybe there are commercial or military applications, etc etc)

Posted in Microsoft, Software | 1 Comment