So, I turn to the interwebs for help: How do I find a summer coding job in Europe? Anyone suggest any companies or individuals to contact, or know of any European coding gigs search sites? Any KDE/Qt companies based in Paris or Italy or anyplace European I should contact? I’ll be in Paris next week traveling, so perhaps I’d be able to meet some folks in person while I’m over there.

This was all fun and games and helped filter out some spam throughout the years, but it seems like it’s time I settle down on one e-mail address. I would also like to find a different domain than zx2c4.com for personal communications that’s much easier to remember (any suggestions?). The problem is – it’s very difficult to switch away from catch-all e-mail. My initial idea was to gather up all the e-mails of individuals who have sent e-mail to something other than my main e-mail address, and automate the process with a form letter sent via SMTP that says something like “Dear joe@shmoe.com, You have sent letters to asdfasf [at] zx2c4.com, thatthang [at] zx2c4.com, jojomojito [at] zx2c4.com. Please note that starting May 1, 2010, only whatIDecide [at] zx2c4.com will be valid. Thank you, Jason”. And then painstakingly go to all of the websites with logins and change my e-mail address for every.single.solitary.one one.by.one. At first this all seemed doable.

But then I started investigating…. Since I started using GMail in March of 2005, I have received e-mail from 1,299 different e-mail addresses on 267 different zx2c4.com e-mail addresses, not to mention lost e-mail from the 5 years prior. I built this python script to give me a good layout of what’s up:

#!/usr/bin/env python
# -*- coding: iso-8859-1 -*-
from imaplib import IMAP4_SSL
from optparse import OptionParser
from email.parser import HeaderParser
from email.message import Message
from email.utils import getaddresses
import sys
import os.path
import os
 
def main():
	parser = OptionParser(usage="%prog --username/-u USERNAME --password/-p PASSWORD --mode/-m MODE [--domain/-d DOMAIN] [--cachedir/-c CACHEDIR]", description="Downloads gmail message headers and determines the set of all e-mail addresses on DOMAIN at which people have emailed you.")
	parser.add_option("-u", "--username", action="store", type="string", metavar="USERNAME", help="Gmail username")
	parser.add_option("-p", "--password", action="store", type="string", metavar="PASSWORD", help="Gmail password")
	parser.add_option("-d", "--domain", action="store", type="string", metavar="DOMAIN", help="Domain name")
	parser.add_option("-c", "--cachedir", action="store", type="string", metavar="CACHEDIR", help="The directory to cache fetched headers for subsequent runs")
	parser.add_option("-m", "--mode", action="store", type="string", metavar="SENDERSFILE", help="If the mode is \"to\", this prints a list of all the emails you've received email on. If the mode is \"from\" this prints a list of everyone who has sent you email. If the mode is \"frombyto\" this prints a list of all the addresses that have emailed you sorted by the address at which you received email. If the mode is \"tobyfrom\" this prints a of all the addresses you have received e-mail from sorted and duplicated by who sent the e-mail.")
	(options, args) = parser.parse_args()
	if options.username == None or options.password == None:
		parser.error("Username and password are all required.")
	if options.mode != "from" and options.mode != "to" and options.mode != "frombyto" and options.mode != "tobyfrom":
		parser.error("You must specify a mode.")
	if not options.username.lower().endswith("@gmail.com") and not options.username.lower().endswith("@googlemail.com"):
		options.username += "@gmail.com"
	if options.cachedir != None and not os.path.exists(options.cachedir):
		try:
			os.makedirs(options.cachedir)
		except:
			sys.stderr.write("Could not make cache dir. Skipping cache.\n")
			options.cachedir = None
 
	imap = IMAP4_SSL("imap.gmail.com")
	imap.login(options.username, options.password)
	imap.select("[Gmail]/All Mail", True)
	typ, data = imap.search(None, 'ALL')
	if typ != "OK":
		sys.exit(("Could not search properly: %s" % typ))
	emailAddresses = {}
	data = data[0].split()
	length = len(data)
	counter = 0
	parser = HeaderParser()
	for num in data:
		counter += 1
		if options.cachedir != None:
			cachePath = os.path.join(options.cachedir, num)
		else:
			cachePath = None
		if cachePath != None and os.path.exists(cachePath):
			message = parser.parse(open(cachePath, "r"))
		else:
			try:
				typ, data = imap.fetch(num, '(RFC822.HEADER)')
			except:
				sys.stderr.write("Failed to fetch ID %s\n" % num)
				continue
			if typ != "OK":
				sys.stderr.write("Failed to fetch ID %s: %s\n" % (num, typ))
				continue
			if cachePath != None:
				try:
					f = open(cachePath, "w")
					f.write(data[0][1])
					f.close()
				except:
					sys.stderr.write("Could not write cache for %s" % num)
			message = parser.parsestr(data[0][1], True)
		tos = message.get_all('to', [])
		ccs = message.get_all('cc', [])
		resent_tos = message.get_all('resent-to', [])
		resent_ccs = message.get_all('resent-cc', [])
		all_recipients = getaddresses(tos + ccs + resent_tos + resent_ccs)
		for address in all_recipients:
			if len(address) == 2 and (options.domain == None or address[1].endswith(options.domain)):
				to = address[1].lower()
				fros = getaddresses(message.get_all('from', []))
				fro_addresses = set()
				for addr in fros:
					if len(addr) == 2:
						fro_addresses.add(addr[1].lower())
				if options.mode == "to" or options.mode == "tobyfrom":
					if to not in emailAddresses:
						emailAddresses[to] = set()
					emailAddresses[to] = emailAddresses[to].union(fro_addresses)
				elif options.mode == "from" or options.mode == "frombyto":
					for fro in fro_addresses:
						if fro not in emailAddresses:
							emailAddresses[fro] = set()
						emailAddresses[fro].add(to)
				sys.stderr.write("[%s of %s]: Message to %s from %s.\n" % (counter, length, address[1], fro_addresses))
		if len(all_recipients) == 0:
			sys.stderr.write("[%s of %s]: Message has empty To header.\n" % (counter, length))
	imap.close()
	imap.logout()
	if options.mode == "to" or options.mode == "from":
		for addr in emailAddresses.keys():
			print addr
	elif options.mode == "tobyfrom" or options.mode == "frombyto":
		for to, fro in emailAddresses.items():
			print to
			for f in fro:
				print "\t%s" % f
 
if __name__ == '__main__':
	main()

The situation seems daunting. That is so many email addresses, so many senders, so many website logins to change, so many people who have to update their address books. So what do I do? What will I do? How important is it to switch away from catch-all? I might just be locked in for life.

I use wpa_supplicant’s optional wpa_gui. It’s a tiny Qt app that has a tray icon and a command line switch to start in the tray. Wpa_supplicant is required for all modern wireless connections and is always running in the background no matter what. Wpa_gui simply connects to wpa_gui’s socket and tells it what to do. I like having wpa_gui in my system tray so that I can reconfigure wifi networks easily.

zx2c4@ZX2C4-Laptop ~ $ cat ~/.kde/Autostart/wpagui.sh
wpa_gui -t

And check it out:

A simple, somewhat ugly, but extremely functional info display. I can connect to new networks with a simple double click:

And presto it connects to the wifi network. I can also configure all of the highly advanced encryption profiles that wpa_supplicant supports. All of this is easily accessible in my tray:

If I did not want wpa_gui -t running all the time, I could pretty easily make this into a little quick launch plasma button, and it would start up nearly as fast, because wpa_gui is so light weight.

This is how I do wireless. I have never had any trouble, and I can connect to wifi networks anywhere I go with ease. It remembers the connections and the priorities that I assign, and I have not seen any system simpler or easier than this.

For wired networking, netplug calls my ethernet setup scripts when I plug in an ethernet cable. No tinkering required. For my cellphone internet via bluetooth, I run “pon nokia” and my ppp chatscript does all the rest. This could easily be tied to a little menu button in my launcher.

I’m bloat free, and networking dynamically on the go with my laptop does not require any advanced timely tinkering.

Why are you all using NM, wicd, etc instead of good ol’ wpa_gui?

The system tray has issues, there’s no support for the disc number, there are some issues with the search filter and multiple fields, I haven’t tested the file renaming capabilities, collection scanning is odd, it still relies on Qt3Support, there is no shortcut handler for scrolling to the current track, and it’s got a lot of unfortunate tiny quirks. But all of this is to be expected of software that’s been basically forgotten about since KDE 3.4.

So after exams are over, I’m going to start investigating Juk’s codebase and seeing if it’s worth it to whip this great player into shape for the prime. It looks promising so far, though I haven’t tested it very completely yet.

The last time I bought a laptop for myself was in 7th grade, 2001. I was 13 and I used all my savings to buy a $3,700 Dell Inspiron 8100 running a Pentium 3 processor with a whopping 512mb of ram, a 64mb graphics card, and a gigantic 60gb hard drive. What an incredible machine this was.

eBay

Finally it’s time I buy myself a new computer. I recently acquired an Alienware M17x that’s pretty high end, but I’m not a gamer, so I’m selling it on eBay and am going to use the cash to finance the purchase of a brand new Dell Studio 1747. This was my first time selling something big on eBay, and at first I screwed up quite a bit — I listed it at $2,200, which is the price I want in the end for it. No one bid (except for a Romanian scammer offering me $3,500), and a friend of mine convinced me to list it at $0.01 – one penny – with no reserve price, and let the market determine its value. I was weary, but so far the auction has gotten to $1,500, which I’m pretty pleased with. eBay is addictive though — I constantly monitor the auction to see how much I’m going to be receiving. It’s taken me over; it’s awful. I must never play the stock market for this reason. To satisfy my obsession with the bidding, I’ve hacked together in a minute a short python script running in screen that sends me a text message every time the bid increases:

#!/usr/bin/env python
import urllib2
import os
import time
last = 0
while True:
        print "Checking auction at %s:" % time.asctime(),
        page = urllib2.urlopen("http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=230409562381").read()
        index = page.find("US $") + 4
        now = int(page[index:page.find(".", index)].replace(",", ""))
        print now
        if now > last:
                os.system("echo 'As of %s, auction is at %s' | sendmail MYTELEPHONENUMBER@txt.att.net" % (time.asctime(), now))
                last = now
        time.sleep(45)

Aye yie yie.

The Specs

But I’m quite excited about the new laptop I’ve ordered:

The specs are:

• Intel Core i7 820QM 1.73GHz (3.06GHz Turbo Mode, 8MB Cache)
• 8GB ram, DDR3, 1333MHz, 2 DIMM
• ATI Mobility Radeon HD4650 with 1GB ram
• Two 500GB 7200rpm hard drives
• 17.3in 1920×1080 Screen
• BluRay Burner
• 4 Year CompleteCare Warranty

With an awesome discount by asking the phone sales manager’s manager’s manager, this all cost before tax $2,350. Still expensive, but I’m hoping that eBay auction will mostly pay for it.

The Interesting Part: Configuration

So while I wait for the new machine to arrive, I have to start planning what I’m going to put on it, and how I’m going to do it.

It has a fast processor and a lot of ram. It seems, however, that it has more ram than processor, in a sense. What I mean is that with 8 gigs of ram, it seems like I might as well keep all programs running at all times. The problem could be though – maybe too many idle processes will gradually eat away at CPU. I’m not sure. Will it?

What do you think? With so much ram, I also need to decide what I’m going to do about swap/hibernation. Do I use a separate partition of 4gigs for swap? Or do I go with the swapfile approach? Or do I not even use a swap at all, because 8gigs of ram is a lot? For hibernation, do I use what’s in 2.6.32 by default or do I patch in TuxOnIce? Decisions, decisions.

I’ll probably go with Gentoo, because it’s customizable and I’ve used it for years. It has its shortcomings, however, and I’m curious about Exherbo, which claims to do what Gentoo does but better, or a customizable binary distro like Arch, or maybe even make the huge leap to an advanced popular distro like Fedora. I’ll probably just stay with Gentoo though, unless somebody can make a really good case to do otherwise. Flamewars, commence.

Of course, I’ll be using KDE SC 4.4 Beta 2 by the time the computer arrives. By the way, I hate adding the “SC” in there, but I think PlanetKDE would come after me if I neglected it. Branding shmanding. But anyway, KDE should fly on the new machine.

The wifi card supports 802.11n, but is unfortunately one of the Broadcom cards that is only supported by the hybrid-proprietary wl driver. Also, in my experience, FGLRX, ATI’s proprietary driver, outperforms RadeonHD by a long shot, so I’ll be using FGLRX too. Shucks. At least ATI’s own John Bridgman has pointed us to a solution for the KWin resizing compositing problem.

The Big Issue: Data Organization

The biggest issue, however, is what I am going to do with 1TB of hard drive space. How do I divide it up? The breakdown of the situation is this:

Storage:

• Internal 500gb
• Internal 500gb
• External 500gb (via USB2)
• Online server with lots of gb (via ssh+rsync)

Data:

• 26gb of pictures (and growing)
• 67gb of music
• 2gb of code
• 1gb of documents
• 15gb of protools archives
• 10gb of goofy home videos from middle school
• 1gb of old school work
• 2gb of middle school junk archives
• 1gb of old code projects
• 300gb of downloaded junk

So what are my options for all of this? First of all, which file system? I’ll probably go with ext4, because it seems like the best you can get on Linux right now. How do I organize the data? Some possible schemes are:

• Many operating systems on Internal1, all data on Internal2, data backed up to both Online and External
• Linux in Internal1 along with main data (pics, docs, music, code), with downloaded junk on Internal2, and archives of old stuff on External, with important stuff backed up to Online
• Linux in Internal1 along with main data (pics, docs, music, code), with downloaded junk and archives of old stuff on Internal2, with important stuff backed up to both Online and External
• Use LVM to concatenate the two drives
• Use LVM for RAID-1
• Use LVM for RAID-0
• Use LVM for RAID-10

What else? What do I do here?!?

I’m really not quite sure. What do you think? What else should I be thinking about? Location of home directory? The additional power consumption of a second HD and hdparm -Y? I want to configure this system perfectly; I need all the help I can get.

Qt 4.6 was released, and Alessandro put together some awesome tutorial videos for setting up a Qt/Symbian development platform.

The installation was not so smooth, however. I have a Nokia 6650, branded and locked down by AT&T, which makes it awful. It’s filled with bloatware I can’t remove and there are all sorts of restrictions in the operating system, especially related to app installation. It would not let me install PIPS, which is required for OpenC and Qt, and gave me either a nasty error from Qt’s demo installer or from the standalone installer an error message like “Component already built-in. You’re screwed.” So I had to jailbreak the phone, by installing HelloCarbide to allow me to install CapsOn/Off to allow me to insert a hacked installserver.exe so that it would let me install PIPS. Whew that took a long time. Debug deployment is also broken with a message like “General OS error.” No clue. It will sometimes work if installed manually, but basically, AT&T’s restrictions make development pretty rough.

Nevertheless, I prevailed in porting my classic Qt example app: MessagePopper. The only modifications were having the main window show with showMaximized() instead of show() for Q_OS_SYMBIAN and focusing a text input, because evidently if a focused widget is hidden and no other visible widget is programmatically focused, the user cannot select anything. The huge addition was adding a function call to ask Symbian for a network connection, and this required including from Qt’s FTP demo the sym_iap_util.h header and qmake pro additions. Oy. I wish this part were built in to QtNetwork.

It worked beautifully though, and here’s a video demonstrating the app:

(link to youtube original)

I did a JS performance test of QtWebKit, KHTML, Gecko, and V8, and here are the results:

Evidently KHTML has a lot of catching up to do, per usual.

• It is clunky. Really clunky. It’s very thick and does not fit into the pocket well.
• The keyboard is too tiny. Maybe this just takes some getting used to, but I couldn’t do it.
• No multi-touch. The little swirling gesture to zoom is a cop-out for a much needed multi-touch screen.
• Unintuitive user-interface. Some things made sense, but generally operations took one too many clicks. Making phone calls should be easy and fast.
• The user interface does not rotate.
• The web browser is turkey slow, even for cached pages.
• Ovi maps is uglier than GMaps, and again – no multi-touch.

Oh well. I guess I’ll wait until the N900+1, especially considering that Maemo 5 will soon be obsolete.

Or maybe I’m just approaching it the wrong way? I dunno. After all, it does run Linux, can run KDE, and is extremely open… but still: it’s lacking heavily in several areas.

So should I wait for the N900+1? $649 is a lot to shell out for a device that is running on a platform that is being retired (“moved to community support”). Besides, I’m a Qt guy, not a Gtk wizard, and any programming I do with the device will most likely be with Qt. Will Maemo 6 be available for the N900? The N900 does not, as far as I can tell, have a multi-touch screen, which means it won’t be able to use Qt 4.6+’s multi-touch gesture support.

On the other hand, how much longer until a Maemo 6 device is released? Probably over a year from now. I also fear, perhaps without good reason, that Maemo 6 (and perhaps even Maemo 5 — I’ll find out later today when I try it out) will have some kind of new-age high-tech web-service-enabled interface that is going to be more difficult to use than my current old school phone. I dunno.

So what should I do? Go for the N900/Maemo5 if it looks decent, or wait who knows how long for the N900+1/Maemo6?

PS: What’s the story with libdui or DirectUI? I can’t find much about this online.

PPS: Is there a future relationship between Plasma and Maemo beyond neat hacks?

Anyone know what’s up?

That is, nothing got done until a lone user piped up with a workaround. Here is what he wrote in the comments of that blog post:

Hi, I have been pissed off by this problem a long time and assumed it was ATI’s fault. Tonight I made one last effort before ordering a Nvidia graphics card. And I was successful.

I am running catalyst 9.8 using a Radeon 3850 and have had this re-size/maximize problem as long as long as I have used KDE4. To solve the problem I needed to modify a file in xorg-server. in the code directory it is called ./composite/compalloc.c. Here I commented out most of a function called compNewPixmap. Everything below these lines:

pPixmap->screen_x = x;
pPixmap->screen_y = y;

all the way down to (but not including) the last line:

return pPixmap;

After this I am running KDE4 with all desktop effects that I want and without any lag in resizing/maximizing.
I am running Gentoo, so I just updated the xorg-server source package file and put it back into the source repository, rebuilt the manifest and emerged it again. Voila!

Voila indeed. The patch he’s talking about looks like this (thanks to this Russian blog I can’t read):

--- composite/compalloc.c.orig  2009-09-08 02:54:28.657143479 +0700                              
+++ composite/compalloc.c       2009-09-08 02:55:42.835357653 +0700                              
@@ -484,64 +484,6 @@                                                                             
     pPixmap->screen_x = x;                                                                      
     pPixmap->screen_y = y;                                                                      
 
-    if (pParent->drawable.depth == pWin->drawable.depth)                                        
-    {                                                                                           
-       GCPtr   pGC = GetScratchGC (pWin->drawable.depth, pScreen);                              
-                                                                                                
-       /*                                                                                       
-        * Copy bits from the parent into the new pixmap so that it will                         
-        * have "reasonable" contents in case for background None areas.                         
-        */                                                                                      
-       if (pGC)                                                                                 
-       {                                                                                        
-           XID val = IncludeInferiors;                                                          
-                                                                                                
-           ValidateGC(&pPixmap->drawable, pGC);                                                 
-           dixChangeGC (serverClient, pGC, GCSubwindowMode, &val, NULL);                        
-           (*pGC->ops->CopyArea) (&pParent->drawable,                                           
-                                  &pPixmap->drawable,                                           
-                                  pGC,                                                          
-                                  x - pParent->drawable.x,                                      
-                                  y - pParent->drawable.y,                                      
-                                  w, h, 0, 0);                                                  
-           FreeScratchGC (pGC);                                                                 
-       }                                                                                        
-    }                                                                                           
-    else                                                                                        
-    {                                                                                           
-       PictFormatPtr   pSrcFormat = compWindowFormat (pParent);                                 
-       PictFormatPtr   pDstFormat = compWindowFormat (pWin);                                    
-       XID             inferiors = IncludeInferiors;                                            
-       int             error;                                                                   
-                                                                                                
-       PicturePtr      pSrcPicture = CreatePicture (None,                                       
-                                                    &pParent->drawable,                         
-                                                    pSrcFormat,                                 
-                                                    CPSubwindowMode,                            
-                                                    &inferiors,                                 
-                                                    serverClient, &error);                      
-                                                                                                
-       PicturePtr      pDstPicture = CreatePicture (None,                                       
-                                                    &pPixmap->drawable,                         
-                                                    pDstFormat,
-                                                    0, 0,
-                                                    serverClient, &error);
-
-       if (pSrcPicture && pDstPicture)
-       {
-           CompositePicture (PictOpSrc,
-                             pSrcPicture,
-                             NULL,
-                             pDstPicture,
-                             x - pParent->drawable.x,
-                             y - pParent->drawable.y,
-                             0, 0, 0, 0, w, h);
-       }
-       if (pSrcPicture)
-           FreePicture (pSrcPicture, 0);
-       if (pDstPicture)
-           FreePicture (pDstPicture, 0);
-    }
     return pPixmap;
 }

This patch works like a charm. All of the FGLRX resizing/maximizing bugs disappear. Not only that, but things like clicking on the K menu are suddenly a lot faster… KDE4 doesn’t seem laggy and now has the performance I’ve expected all along. The effects look great, and my transparent terminal is a delight.

There is, however, a bit of garbage that shows up occasionally, and perhaps there’s a good use for the code that was removed in the patch. Why is it only FGLRX that benefits from removing this code? I don’t know much about XOrg internals, but I’m guessing it has to do with some sort of sometimes-required allocation that causes a readback in the FGLRX driver but not in other drivers. What’s the deal? Is fixing this problem as simple as committing this patch and then fixing the garbage error? Or is the code that was removed necessary, and really the problem lays with FGLRX? What to do at this point?

Evidently KMix is supposed to support PA, but it still only shows the alsa devices. I suspect this is related to when PA is loaded, and currently I do not have any user-space loader. What’s the optimal way for this to happen in KDE? An autostart entry? Symlinking some mysterious file to kde’s env directory? All the forum posts are for older KDE versions. Phonon works so far though. All the PA apps are GTK/GConf based, and aren’t very KDE friendly, and generally it seems like KDE has neglected PA support, or PA has neglected KDE support. They simply aren’t very pretty together. What are the plans for integrating KDE and PA a little bit more closely?

I’m using a Macbook as a print server, printing to it from my Linux box, and everything is simple because they both use CUPS. However, I’m having a bit of trouble streaming audio to it. I can’t seem to build PA on OSX, and I don’t even think OSX is officially supported by PA. I tried installing ESD using MacPorts and using PA’s bridge for that, but it played a half a second of sound before skipping. ESD doesn’t do good latency calculations. PA supports streaming to Airport Express, so I thought I’d try out Airfoil Speakers, but unfortunately, Airfoil uses a different protocol. So I’m not sure what to do now… How do I send PA to my Macbook?

PA also doesn’t work well with Skype, and I anticipate some other problems as well. What a hassle. Why the bad integration with KDE? Why the numerous bugs? Any PA tips from veteran readers?

I capture the downloads by using QtWebKit’s unsupportedContent signal and parse the downloads by using QNetworkAccessManager’s finished signal. I then run the zip archive through QtIOCompressor’s code for unzipping zips, then extract and parse the subtitles.

A significant issue is that a lot of the subtitles on Subscene are from torrents that have videos which are cut off at the beginning. The solution is to shift each subtitle in the file by a constant time interval if neccessary, so I built an interface to easily see and shift timing:

As you can see, it’s possible to shift all the subtitles. Phonon made this page a breeze. One significant challenge was writing a search algorithm that would find the correct Subtitle based on a time falling in a certain range. It needed to be super speedy and designed for sequential searches. I came up with the following, which could use a bit of improvement:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

QString SubtitleParser::subtitleAtTime(quint64 time)
{
	QLinkedList<Subtitle>::const_iterator i;
	bool forward = true;
	for (i = m_searchPosition; i != m_subtitles.constEnd(); ) {
		if (i->isContainedIn(time)) {
			m_searchPosition = i;
			return i->text();
		}
		if (i->isAfter(time)) {
			if (i == m_subtitles.constBegin())
				break;
			--i;
			forward = false;
		} else {
			if (!forward)
				break;
			++i;
		}
	}
	return QString(' ');
}

Another challenge was dealing with QWizard and QWizardPage, particularly issues with fields and custom buttons. The field/property system is just too cumbersome to work with; there ought to be a better way of establishing centralized data with straight up pointers. The buttons on QWizard all want to be global, so adding a custom button to a single page proved a bit hackish. It’s a cool API, but just a little bit limiting. If anyone has some suggestions, let me know.

As always, the code is available on my gitweb, and I encourage you to try it out, read the code, and send me suggestions in the comments below.

Though this was designed to work on Linux, Qt, being delightfully cross-platform, also makes it possible to run the Subtitler on OSX. Check out the OSX screencast:

Direct YouTube Link

Amongst users of the proprietary ATI driver (FGLRX), it’s a well known problem that KDE4’s KWin’s desktop effects are unusable due to turtle slow window resizing. Unfortunately, bug reports and forum posts fail to come up with a clear answer as to why this happens, who is responsibility for fixing it, and which parts of the stack need to adapt for the bug. If you’re not already familiar with the problem, observe:

Direct YouTube Link

As you can see, the three second delay in window resizing, maximizing, and (perhaps) restoring makes compositing-enabled KWin unusable. Note that the problem still persists when disabling the “show window content while resizing” option.

It’s been particularly difficult to track down the route of this problem and who should be fixing it. From a lengthy discussion on Phoronix’s Forums, ATI’s “Bridgman” user acknowledges the problem but claims “It’s not a driver issue AFAIK, so not sure why we would mention it in the release notes.” He goes on to indicate that the problem was introduced during a hack to ensure KWin compatibility on Intel cards, to work around a glitch with garbage appearing on the screen, and a lengthy debate over the “107_fedora_dont_backfill_bg_none.patch” patch. Somewhere the stack was patched to support Intel cards at the expense of ATI cards, according to ATI’s “Bridgman”.

Over on the KDE bugzilla, we’re met with a curt response from the KWin team: “Driver bug then.” The bug is marked as “RESOLVED” and “UPSTREAM”.

And elsewhere on the Internet, we get a helpless string of “still not fixed” messages, over on the unofficial FGLRX bugzilla, other Phoronix forum posts, and distro forums.

So I ask my readers: who is responsible for this bug and how will it be fixed? If indeed it is neither FGLRX nor KWin and the problem exists in XOrg, why has XOrg been patched to support Intel cards at the expense of ATI cards? Is there anything FGLRX or KWin could be doing to mitigate the problem? Potential workarounds? Users have complained of this problem for over a year now perhaps, but not once has leadership from anywhere offered a compelling explanation or plan for the future. What to do?

Leave comments. The current discussion seems to be fruitful.

AnyClip will launch later this year as a comprehensive library of legally-licensed scenes from Hollywood movies. While we’re still in active negotiations with the rights holders of the films, this preview-only Boxee application begins to show the potential behind tying such a powerful platform to applications like Boxee. Ultimately, applications like AnyClip for Boxee will enable people to relive any moment, from any film, on any platform.

I submitted it to the Boxee Dev Challenge, and polls are open for voting. Please vote for AnyClip here.

It is also (nearly) possible to emulate the layout of Juk, iTunes, Banshee, or Rhythmbox:

It is not now configurable and is rather inefficient because getlisting.php does not support very complex queries (though I do take advantage of its sorting on lines 54-57), but this will be changed by the time I release an installable version of the script. Currently, it downloads a listing of the entire collection during the first request (lines 27-33), and then just reads from an in-memory database (multidimensional array) to query the different levels (lines 34-95). TMI: too much iteration.

Unfortunately there are some kinks, as Phonon often has trouble playing URLs and does not always show buffering information correctly (both gstreamer and xine backends). Look at lines 65 and below of getsong.php: this should be a working HTTP streaming implementation, but Phonon doesn’t dig it, for whatever reason. Any pointers here? The same issue crops up in the standalone qt player for ZX2C4 Music. Strange. Also, I use QTextDocument to convert HTML entities back to normal text (lines 14-22), but it’s very slow. Is there a better way to be doing this? Finally, callbackData (line 73) refuses to store an array, which means I have to use a nasty splitter; this is a bug presumably.

For the ZX2C4 Music hackers out there, here’s the source code to play with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99

Importer.loadQtBinding("qt.core");
Importer.loadQtBinding("qt.network");
Importer.loadQtBinding("qt.gui");
 
function ZX2C4Music()
{
	ScriptableServiceScript.call(this, "ZX2C4 Music", 3, "Browse ZX2C4 Music", "The entire ZX2C4 collection, at your finger tips.", false);
}
var songs = null;
var delayedArgs = null;
function receiveDatabase(reply)
{
	songs = eval(reply)[1];
	var decoder = new QTextDocument();
	for (var i = 0; i < songs.length; i++)
	{
		for (var j = 0; j < songs[i].length; j++)
		{
			decoder.setHtml(songs[i][j]);
			songs[i][j] = decoder.toPlainText();
		}
	}
	onPopulate(delayedArgs[0], delayedArgs[1], delayedArgs[2]);
}
function onPopulate(level, callback, filter)
{
	if (songs == null)
	{
		delayedArgs = [level, callback, filter];
		new Downloader(new QUrl("http://music.zx2c4.com/getlisting.php?username=TOP&password=SECRET&language=javascript"), receiveDatabase);
		Amarok.Window.Statusbar.shortMessage("ZX2C4 Music collection is loading...");
		return;
	}
	if (level == 0)
	{
		var splitCallback = callback.split("&*/ZX2C4MUSICSPLITTER/*&");
		var notFirst = false;
		for (var i = 0; i < songs.length; i++)
		{
			if (songs[i][4] == splitCallback[0] && songs[i][3] == splitCallback[1])
			{
				notFirst = true;
				var item = Amarok.StreamItem;
				item.level = 0;
				item.itemName = songs[i][2];
				item.playableUrl = "http://music.zx2c4.com/getsong.php?username=TOP&password=SECRET&transcode=false&hash=" + songs[i][0];
				item.album = songs[i][3];
				item.artist = songs[i][4];
				item.track = songs[i][1];
				item.infoHtml = "";
				item.callbackData = "";
				script.insertItem(item);
			}
			else if (notFirst)
			{
				break;
			}
		}
	}
	else if (level == 1)
	{
		var lastAlbum;
		for (var i = 0; i < songs.length; i++)
		{
			if (callback == songs[i][4] && lastAlbum != songs[i][3])
			{
				lastAlbum = songs[i][3];
				var item = Amarok.StreamItem;
				item.level = 1;
				item.itemName = lastAlbum;
				item.playableUrl = "";
				item.infoHtml = "";
				item.callbackData = callback + "&*/ZX2C4MUSICSPLITTER/*&" + lastAlbum;
				script.insertItem(item);
			}
		}
	}
	else if (level == 2)
	{
		var lastArtist;
		for (var i = 0; i < songs.length; i++)
		{
			if (lastArtist != songs[i][4])
			{
				lastArtist = songs[i][4];
				var item = Amarok.StreamItem;
				item.level = 2;
				item.itemName = lastArtist;
				item.playableUrl = "";
				item.infoHtml = "";
				item.callbackData = lastArtist;
				script.insertItem(item);
			}
		}
	}
	script.donePopulating();
}
var script = new ZX2C4Music();
script.populate.connect(onPopulate);

Update: Possibly this coming week we’ll meet. Anyone know of good coffee shops in Manhattan that have wifi? Or perhaps I can procure a spot in a Columbia building…

I’m just a student, but I can build just about anything with code. The problem is that people in the industry disregard me because I’m young. I also doesn’t have many “contacts” or “networks” for shmoozing up. I am, however, looking for a lucrative freelance coding job for this summer.

So I’m wondering – amongst those of you who have been in a similar predicament, how have you gone about finding programming gigs?

A related question: does anybody hire KDE programmers?

Sincerely,
Unemployed

I am curious about the future of KHTML in the face of WebKit. Three questions come to mind:

1. Why does KDE use KHTML when WebKit is faster, more compatible, etc?
2. Why does Konqueror use KHTML as default instead of WebKit?
3. Will KHTML be moved out of the main tree?

As far as I can see, it is commonly accepted fact that WebKit is a faster and more compatible engine than KHTML. WebKit was ported to Qt in 4.4, and Qt 4.5 provides some critical enhancements. From these two points, the three questions above naturally follow. If the situation for KHTML was completely hopeless, then it never would have made KDE 4.0 or the present API would be converted into a wrapper for QtWebKit. But this is not the case, so presumably the KHTML team has high hopes for the project. I wonder what the team’s response is to these questions. Undoubtedly they have thought a lot about KHTML vs WebKit and find WebKit a worthwhile project. What’s the deal?

Update:
There will be a KDE 4.2 release event at the Hungarian Pastry Shop on 1030 Amsterdam Avenue (between west 110th and 111th) at 8pm on January 28th. Visit wiki information.

Prior to the fix, only one php page could load at a time. When requesting a second php page while one was loading, the request would hang until the previous one had completed. This is not terribly problematic for ordinary web pages, but because I use php for file downloads, the entire web app would hang until the file had completed downloading. Bad news bears. After a lot of difficult debugging, tcpdumps, header inspects, ps aux, etc, I discovered the problem to be with session_start(). It turns out that only one session can be opened for writing at a time per session cookie, which means for a single user, php will only allow pages to write to his session one at a time. The solution was to call session_write_close() immediately after editing the last $_SESSION variable.

Some media players had trouble seeking in streams from the php file streamer because of bad http header range support. The sendFile() is now HTTP/1.1 compliant. Although, I’d like to add some caching support at some point.

Finally, and most notably, ZX2C4 Music no longer hangs when loading giant lists. It now loads 50 items at a time, and polls the location of the scrollbar every 100 milliseconds, and if it is close to the bottom, it requests the next 50 items. Polling is not ideal, but JavaScript does not have scroll events for divs. The pagination is implemented using MySQL’s LIMIT and OFFSET features. The display code took quite a bit of reworking and might still be a little buggy. Do try it out. Internet Explorer requires a special case because of its lack of support for writing to tables’ innerHTML, and Microsoft refuses to fix it. When adding an entire list to the download basket, it takes the time to download and display every item in the query, like the old behavior, and this may introduce some bugs. For the most part though, scrolling auto-pagination support is generally pretty slick.

The download basket/zip feature is no longer capped at 200 songs, as it no longer needs to call an external zip program and store the zip file temporarily. In pure php, I’ve written a custom on-the-fly zip file streamer, which works a lot faster.

As always, if you have suggestions, leave a comment below. If you’d like to download the latest HEAD of ZX2C4 Music, you can browse the repository here or download a tar of it here. Next on the list are HTML5 <audio> element support for web app streaming, Amarok plugin, update script, and maybe a revamped interface.

The iterative approach to this problem is easy, but I am interested in determining a non-iterative solution to finding all of the O_i, where I simply have a formula O_i = [yada yada yada].

Here’s code for the iterative solution when applied to columns of a QTreeView, with N = 3 and R = .42, which is derived from my AutoSizingList class of ZMusicPlayer. It uses QTreeView’s sizeHintForColumn to determine M_i.

So essentially, the problem is that the do-while on lines 18 and 41 should be eliminated. It is required in the first place because when proportionally redistributing the difference between the arguments of the min function (outlined in the first paragraph), the ratios of the columns receiving this extra width may exceed R, which means another iteration must be done to readjust. Anyone have a non-iterative solution? Also, to the Qt experts out there, why is the hack required on line 49?

I am interested both in a practical Qt approach and a mathematical approach, the latter being more interesting. Doesn’t this seem like functionality that aught to already be a part of Qt?

Update: A conversation with a friend over AIM has produced a plain-English explanation in the form of a dialogue of the above problem.

Update: source code available here.

I have just finished generalizing aspects of ZX2C4 Music so that it’s installable on a variety of different servers. I have no experience with releasing PHP apps to the general public, so I’d appreciate some feedback on the installation process. You can download the tarball here. Be sure to read README.txt.

A few months ago I switched website hosts from Netfirms to Site5, because Site5 had a good deal for a plan with unlimited bandwidth and disk space. The first thing I did was upload my entire music collection.

To access all of my music remotely via any internet-connected web browser, I wrote ZX2C4 Music, which is password protected to avoid the law, but if you’d like to view the interface just for curiosity, ask me in the comments section, and I’ll e-mail you a password. It uses PHP/MySQL for managing the music, hashing each file with an sha1, TagLib for reading tags, and JavaScript/HTML as an interface:

Search queries are done with AJAX, and playing is done with Flash, if available; otherwise it defaults to browser plug-ins. Soon I should add HTML5’s <audio> element support. Since Flash only supports MP3, the php backend optionally transcodes non-mp3 songs on the fly by using ffmpeg. Songs can be downloaded in zipped bundles by using the download basket feature. The web interface still needs some work, most notably scrolling-pagination for the song table, but it’s definitely usable. Many have pointed my attention to Ampache, but I found it too big and overwhelming. You can view the source code here.

Of course, the next step was writing a Qt or KDE application that interfaces directly with the web backend using Phonon. Meet ZMusicPlayer:

It downloads and displays a compressed and sorted xml listing of all the songs and their sha1s. The columns of the list automatically size optimally using an awesome algorithm that still needs some tweaking (why is subtracting 40 required on line 87?). Suggestions?

It plays the songs by passing the appropriate QUrl to Phonon, and this is where the trouble begins. The xine backend on KDE 4.1.2 just skips one song to another most of the time, and when it does miraculously play, seeking (via HTTP partial requests) is unavailable. The headers for each of the server responses for song file requests provide the appropriate parameters for HTTP seeking. Maybe this has already been fixed in trunk, but if not, I aught to investigate the problem as soon as I switch my 4.1.2 desktop completely to trunk. GStreamer with Qt 4.4 allows for proper streaming if a GStreamer HTTP plug-in is installed (gst-plugins-good includes a wrapper plugin for libsoup, which works, for those who don’t want to install the gnome vfs plugin), which is understandable. However, seeking only works for mp3 files; m4a files play but cannot seek, even though the pertinent response headers are the same for both file types. On the Windows backend, the seeking situation is hopeless, and on the OS X backend, songs do not play at all. Also, xine is the only one that emits the proper buffering signals. On some setups though, some of these problems go away. I still need to do some more thorough debugging.

If you’d like to download and build ZMusicPlayer, you can browse the source, clone the repository with git clone http://git.zx2c4.com/zmusicplayer.git, or download a tarball. Depending on your package setup, you may have to tinker with the includes for the phonon headers. I’d love to hear some suggestions, so leave comments below. Once I stabilize the Qt code, I’d like to develop some optional KDE extensions, particularly integration with the KDE’s keyboard shortcuts.

Also: I’ve switched from blogging on kdedevelopers.org to my own Wordpress blog so that anyone can post comments without requiring an approved account. So, comment away. If you wanted to comment on my first post, but were unable to, you can now comment here. To comment on this post, use the link below.

Update Oct 25, 3:31pm: Several of you have e-mailed me to say that the browser hangs when loading a unfiltered giant collection. This is indeed the case, as no current browser is able to write 10,000 rows to a table in a reasonable amount of time. This is why above I speak of “scrolling-pagination”, which means that the table would load 100 or so rows at a time, and then auto-populate the remaining rows in 100 row increments every time the user scrolled to near the end.

Speaking of projects, to learn more about Qt’s painting system, I wrote a program that generates the Sierpiński triangle using a chaos game. Pick three vertices of a triangle. Plot a point at random inside that triangle. Choose a vertex of the triangle at random and plot the midpoint between the chosen vertex and the previous point plotted. Repeat this ad infinitum and you get:

I don’t think that I’ve implemented the drawing in the most ideal way, so please leave suggestions. I opted to use a QImage instead of a QPainter, and then just fill pixels and scale. You can browse the source, checkout the repository with git clone http://git.zx2c4.com/trianglefractalchaos.git, or download a tarball.

Currently I’m running 4.1.2, but I plan to switch completely over to trunk in the next few weeks. Here’s my default clean desktop:

The large note on the left is about a few ideas for a shared-server-plan based file sharing mini-network. Maybe I’ll say more on this another time if I end up implementing it.

Hello planetKDE.