One Year Anniversary

Dear Gord, has it already been a year since I started this project? That's even lazier than I usually am. No, the project isn't abandoned (not quite); though I guess I'm lucky the project didn't get usurped while I've been off in my own little world.

I've finished cleaning up all of the code for the patcher DLL (the core of MPQDraft) and the SEMPQ stub code, and both are now in the repository. Unfortunately, as the GUI code isn't cleaned up yet, those parts aren't particularly useful apart from simply looking at how it works. I may have to start considering uploading the rest of the code immediately, and suffering through various laughing and insults for the icky code...

I've also added MPQDraft builds made with the code in the repository now and the code that hasn't yet been added, for testing. I did a substantial amount of code cleanup, and some portions were rewritten from scratch. I did some basic testing to make sure that it seemed to be working more or less, but it wasn't exhaustive, and I've only tested it with a couple of my own plugins.

Open-Source MPQDraft

For the Love of Kaity...

Recently, it has been observed that Comcast is disrupting TCP connections using forged TCP reset (RST) packets [1]. These reset packets were originally targeted at TCP connections associated with the BitTorrent file-sharing protocol. However, Comcast has stated that they are transitioning to a more "protocol neutral" traffic shaping approach [2]. We have recently observed this shift in policy, and have collected network traffic traces to demonstrate the behavior of their traffic shaping. In particular, we are able (during peak usage times) to synthetically generate a relatively large number of TCP reset packets aimed at any new TCP connection regardless of the application-level protocol. Surprisingly, this traffic shaping even disrupts normal web browsing and e-mail applications.

New traffic shaping can disrupt a Comcast Internet connection

I think I hear the entire Comcast tech support department committing seppuku.

From a technical standpoint, there are few options that are more idiotic than sending reset packets to kill BitTorrent connections, as Comcast was doing previously. They just did one: killing ALL connections in that manner. This option is so bad, in fact, that it leads me to seriously consider the possibility that Comcast is doing this intentionally to teach the FCC a lesson: that its resetting BT connections wasn't so bad. What could/should Comcast have done differently? Let's look at a few possibilities.

The best option (excluding improving their infrastructure) would be to monitor the amount of traffic going through each modem, and if a disproportionately large amount is coming from one modem, tell the modem to limit traffic rate for that one modem. Unfortunately, I'm told Comcast modems do not have the ability to change maximum speed without a reboot, so this isn't really possible.

Failing that, there is an extremely simple yet efficient method: simply start randomly dropping packets when there's congestion. While this may sound like a sarcastic suggestion, it's not. The TCP protocol uses two pieces of information to determine how fast to send data: the amount of buffer space the receiver has (used to prevent the sender on a fast connection from flooding a receiver on a slow connection), and dropped packets. If a dropped packet is detected, TCP lowers its send rate. And if BT is a large proportion of traffic, randomly dropping traffic will result is a larger number of BT connections getting throttled than non-BT connections.

Thus this is a simple and easy way to effectively lower the amount of data being sent in a way biased against heavy bandwidth users, without interrupting any of the connections. This works even better if their Sandvine hardware can detect BT traffic and selectively drop packets only from BT traffic. As long as they weren't dropping so many packets that BT slowed to a crawl, I wouldn't mind my ISP doing this.

Sidebar Addition

For those who actually watch anime, I've added a list of the anime I'm currently watching to the sidebar on the right. I'll have to remember to update it as the info changes.

New Job

So, with a few weeks left to the school semester, and starting to work on my three term projects, I've got a lucrative new job lined up: a reverse-engineering and programming position at MediaSentry/SafeNet. For those wondering, Q's soul is worth $140,000/year.

Exercise for the Reader Solved

Or rather, my conclusion about the answer to this question on the test (I posted the question to see if anyone else could figure it out) was confirmed by the teacher: there is no way to reduce the storage of any set of arbitrary points below the 96 bits required for the raw coordinates. There was an error in the question; specifically, the teacher forgot to mention one additional constraint about the values of the coordinates (though he hasn't said what it was, yet).

As for possible solutions. If it were known that all points were coplanar, you could store the normal of the plane and first point (or the first three points themselves), then convert all other points to 2D coordinates. You could use any manner of 2D coordinate system (they'll all use the same amount of space to store), but as we know the first three points are not collinear, I'd say the easiest would be barycentric coordinates. This was the first thing that came to mind while I was taking the test, but I asked and he said that I couldn't assume all points were coplanar.

Alternately, if you knew something about the locations of the points with regard to the plane, such as that they formed a regularly-spaced grid, or occurred periodically on a curve, you could reduce each point to a single value - the distance from the plane - with some overhead for describing the pattern the points appear on. Given that the hint the teacher gave me after I had finished the test was "height map", I'm guessing this is what he had in mind.

What's the Matter with MediaSentry?

Let me attempt to give a brief sketch of how P2P applications work, targeted toward non-comp-sci people, particularly with respect to the RIAA file sharing suits. This is taken from the general knowledge I have about P2P programs as a computer scientist, and in some cases I know specific things about specific applications.

Basic Architecture
In P2P applications, individual computers act as either clients or servers (or both at the same time), depending on whether they're downloading, uploading, or both at a given time. Somewhere there is a list of computers in a given P2P network - which users must join at some point. For eDonkey, you log into a large network when you start the program, and these large networks contain many peers who may never communicate with each other. With BitTorrent, you are logged into a small network - only containing the files you are downloading/uploading (e.g. MP3s of a single CD) - only while you are downloading/uploading, and you may be connected to many such networks at once.

Somewhere there is a directory of users. This may be stored on a single server (I believe eDonkey was this way), or a network where a computer asks another computer "who have you seen recently?", and then asks that question of all the computers that are returned from the query, etc.

In all cases (unless you've got some kind of file-sharing virus, which I'm actually surprised we haven't seen before), the user voluntarily logs into and out of the network(s) through various actions. As well, files which are shared must be "voluntarily" shared, either from a shared files folder or by tracking specific files that should be shared; however, most programs will automatically share any files you download from other users (and I've heard some programs, when installed, automatically search for and share files that the program thinks would be good to share).

Depending on the system, peer and/or the directory server may not know all the files a given computer has available for sharing. Similarly, if you ask a given computer what files they're sharing, it may or may not be a complete list. In torrents, it only shows the files in the particular torrent; I believe eDonkey lists all shared files. All P2P systems have a way of asking a particular computer what files they are sharing, although the completeness of the response varies.

Okay, getting to the specific legal issues.

Method of Obtaining an IP and File List
First, as there are standard and intended methods of asking a computer what files it's sharing, it's (probably) not true that MediaSentry had to do anything illegal to obtain this list, like hack into the computer. Likewise, they probably didn't have to do anything other users couldn't do (although they probably made a program to scan P2P networks and catalog all files, while the typical user would have to search for people with specific files; I wouldn't call this illegal).

The big question mark is how exactly MediaSentry verified (to the best of its knowledge) that the info they obtained is true, and without knowing this we can't give a good estimate of the false-positive rate (which is likely the reason MediaSentry won't say what their methods are; they're probably lying when they say that they have developed proprietary and novel methods of investigation that should be considered trade secrets), although previous cases have shown this rate to be > 0. There are lots of ways an investigation could go wrong (or become difficult), even if they did see what appears to be a computer sharing copyrighted files.

Outdated Cache Information
It's possible that the directory server or another computer has an outdated list of files shared by a certain computer, in which case they may say that a computer is sharing files that it isn't. One example of how this could happen is that a computer was sharing some files on some network then disconnected from the internet, and another computer logged on and was given the same IP. Such outdated data could indicate that the second computer is sharing files, even though it's not (it might not even be on the P2P network at all), and in fact NOBODY at that IP has been sharing files for some time. This goes directly to the issue of not being able to positively identify a person from an IP address even if you get an IP address that that computer has as the moment the IP address is obtained (although this is highly dependent on the P2P program). This risk of false positives (and the next one or two) can more or less be eliminated by verifying that the files can actually be downloaded at the time the IP is seen "sharing" files.

Leeching
It's possible that the user is a "leecher" - somebody who downloads without allowing their computer to upload anything by messing with their system configuration. This may be done either intentionally (it's not extremely rare for people to leech so they don't have to use upstream bandwidth when all they want is to get something from someone else; such people fall under the "jackass" category) or unintentionally (P2P programs can be a huge pain to set up to work properly when you're behind a home or other type of local network, and even some ISPs block P2P uploading - but not downloading). Obviously if they're a leecher, they haven't so much as made available anything, despite the computer indicating that it's sharing stuff (although intent becomes a big question if they're not intentionally leeching). While some P2P networks will ban leechers, it's possible that leechers can report false info to the server for the explicit purpose of evading leecher banning; consequently, leeching must specifically be ruled out by successfully downloading the "shared" files.

Clock Synchronization
The issue of stale data comes up again at the ISP and organization (if there's a large network such as a school that the violating computer is on); though more importantly, there's no guarantee that the clocks on the MS computer (here I'm assuming they've actually downloaded the files from the sharer) are synchronized with the clocks on the ISP/organization. If these clocks aren't well synchronized, there's always the possibility that the account information they get from the ISP/organization isn't for the account that had the IP at the time the files were shared. This would require explicitly testing clock synchronization between everyone involved; I'd imagine it would be troublesome to get an ISP/other organization to put that kind of effort into a response to a subpoena. Although this possibility can alternately be reduced by the ISP/organization checking that there are no logons near the time sharing supposedly occurred; if there is a very large area where no logons occurred, the probability of a false positive is probably negligible, even if the clocks aren't precisely calibrated.

Network Address Translation
Next, NATs provide a major problem for identifying the offending computer, because it's entirely possible that that there are multiple computers using the same IP at the same time. In theory (and subject to the problem in the next paragraph) the router can distinguish which computer has which connection at what time (NATs assign unique port numbers to each computer sharing an IP address), but the probability of this information still being around by the time a suit is filed is low, even under normal (non-destruction-of-evidence-type) use. Whether an IP is a NAT or a single computer can be halfway reliably determined by investigators like MS using public info (I recall you got hung up on that point in one of the early trials of yours). If the IP is a NAT, it's going to be significantly harder to prove which computer shared the files, and requires forensic examination of the hard drives or someone on the network confessing (or the RIAA's preferred method: file a suit against the account holder and expect them to give up the person responsible rather than face court or settlement costs). However, this problem is short-circuited if the RIAA gets lucky enough that the P2P application uses user names (some do, some don't), and the name of the sharer is known to be used by a certain person (although I suppose someone could maliciously use the name of somebody they don't like).

IP Spoofing/ARP Poisoning
As well, I'm told by people more knowledgeable than me (I came up with the idea, and then asked them to verify that it could be done in real-world networks) that depending on the configuration of a network it's possible to operate under the IP address of somebody close to you (perhaps somebody in your dorm). This would very likely require intent to deceive, but it might be attractive for someone who wants to download stuff without getting in trouble. I don't know if there are tools out there that make this easy enough for your average user to do, but it's definitely technologically possible, given the right network. In fact, I have a friend who is a very skilled network "hacker" (he publishes articles in security journals) who had written a program to disconnect file sharers from his school network because they were hogging bandwidth and making his connection slow (and he did so without being a network administrator, as far as I'm aware; however, that was a simpler case than sustaining two-way communication); network hacking is outside my field of expertise, but I'm betting this involved what I'm describing in this paragraph. it would depend on how secure the network configuration is, but I'd reason (keeping in mind that I know some about networks but they aren't my specialty - I'm just highly inquisitive, and know a moderate amount about many topics) that wireless networks are especially vulnerable to this. Ruling this out requires knowledge of the physical layout of the network the sharing computer is connected to, and the network administration policies (I would guess that this is usually not done due to annoyance for the network administrators, but perhaps some would). This seems like a viable defense, but I'd recommend talking to a network expert about this directly (my friend isn't online at the moment, so the confirmation of feasibility didn't come from him).

Making Available
Finally (at least I think this is the end), there's the nebulous issue of making available. Even if MS knows for sure that this person was on this computer with this IP address at the time, and MS successfully downloaded a valid copy of a copyrighted file, there isn't a guarantee that this file was actually distributed to other people. In P2P applications with very large networks, it's very possible that simply nobody other than MS ever asked for a copy of a file from a specific computer, so there was no actual distribution. In such cases, it becomes very difficult to even estimate the probability of someone else downloading the file (as I've explained that it's not enough simply to ask other computers if they downloaded the file from that computer - even if the P2P application has a way of asking that - as the computer may be lying or propagating incorrect data (it could be that the "sharing" computer is a leecher and only says it uploaded the file). Obviously this is only an issue if making available is not ruled to be equivalent to distribution.

Questions, comments? Anything (or everything) I didn't explain well enough for laymen, or anybody technically apt want to know exactly what I'm referring to in some cases (it might not always be clear exactly what I was referring to, as I didn't explain the technical details behind that list of risks)?

And oops, I did it again - sat down to write something that was supposed to be fairly concise, and ended up writing something that looks like a judge's ruling document. But at least it made me forget about my flu for a couple hours, so that's a good thing.

Random Japanese Fact of the Day

Rather than have many particles that indicate relative position, such as English 'on [top of]', 'to the left of', 'above', 'in', etc., Japanese uses nouns for these things - 'top' (上), 'left' (左), 'inside' (中), etc., and has only a single word to indicate place (actually two - 'に/ni' and 'で/de' - with the context determining which is appropriate). An example from a CD booklet:
楽曲群 [set of compositions/music] の 中 [inside/medium/middle/among] で

This would literally translate to "at the inside of the music", or, more freely, "in the music".

It's a pretty elegant system. And this is of particular interest to me because of the fact that there's a clear hierarchy of word classes (in terms of importance) in Caia:
1. Nouns
2. Verbs
3. Everything else (adjectives, adverbs, particles) except conjugates (I don't really know how to rank conjugates in importance)

Exercise for the Reader

Here's a question right off my graphics programming test today:

You have n (greater than 3) 3D points, each represented by three 32-bit floats. The first three points a, b, and c are guaranteed to not be colinear, but nothing else is guaranteed. Devise a strategy to store the set of points in minimal space (hint: use a plane defined by a, b, and c), and give how much space would be needed to store the n points.

Agglutinative Train Wreck

While looking up some other Japanese words, I ran across this one:

ＡＢＣ順に 【エービーシーじゅんに】 (adv) in alphabetical order; ED

ABC is obviously an example of the meaning of the word, 順 means order, and に is the locative particle (in this case having the meaning of "in"). I thought it was hilarious that there's a Japanese word that contains Chinese kanji, Japanese hiragana, and Roman characters all together.

& AI Class 2 - Initial Plans

So, that midterm went badly. Between the fact that I didn't study at all (it was an open-book test), and thus took longer than would be ideal, and the fact that I forgot to bring the single most important piece of paper I have (the LISP function reference sheet), this won't make my list of highest test scores ever. On the plus side, I got some feedback on my ideas for a term project (the ones I posted) from my teacher, chose one candidate to submit as a proposal, and got approval for it today.

One experiment described in my AI textbook involved speech synthesis and machine learning. An AI system took in a letter from an English word, along with several letters before and after it, and produced a phone - the precise spoken sound - for that letter. The experiment constructed two separate implementations, one based on the ID3 symbolic learning algorithm, and the other based on back-propagated neural networks, and compared the performance and characteristics of both implementations. Both are trained by feeding in streams of examples where the input and output are both known, and the machine learning algorithms adapt the actual output of the system to most closely match the correct answers.

Using this as a model (partly because it's vaguely similar, partly because it was just a convenient model), my experiment is to construct a system which identifies the language a piece of text is in, based purely on dumb pattern recognition rather than any specific knowledge about the structure of the languages (not unlike the model experiment). How exactly I intend to accomplish this (or at least attempt to) is where things become nontrivial, although I'd be lying if I said anything I'm going to do is particularly hard.

In this project, I intend to create multiple systems based on the same learning algorithm, one or two per language, each returning a single boolean indicating whether the algorithm thinks the current input is something in its language. The decision was motivated by the fact that a single composite system, while likely more accurate, would depend on input from ALL languages. Changing the set of training words in one language would affect the output for all languages, making it a nightmare to test incrementally.

There are three general levels of structure to language that a dumb system might be able to recognize: phonology, morphology, and syntax. Phonology describes the sounds in a language and in what order they may appear in the language (in our case, where we're using written material rather than spoken, replace phonology with orthography - how a language is written, which is related to phonology). Morphology is how words are constructed and modified. Finally, syntax is the order words appear in.

I intend to base detection on orthography and possibly syntax. Both are relatively easy to evaluate, while morphology is much more difficult (at least for my level of skill). In both cases the basic idea is the same: the program iterates over units of text, testing each one through the AI function, and counts the number of matches. It then compares the number of matches between the functions of the different languages with various statistical functions to attempt to determine if there's a clear conclusion. Exactly what statistical methods to use will probably require a fair amount of experimentation.

In the neural network implementation of both cases, this will require construction of a common character set for all the languages used, as inputs will be binary; in other words, there will be many inputs - one per character in the character set per character in the sample. This would (likely) make it infeasible to support even UCS-2 (one flavor of Unicode), as that would require hundreds of thousands of inputs. I'm expecting the combined character set to be around 35-50 characters.

Because of the limitation on character sets (and the obvious fact that character sets alone could be a dead giveaway in some cases, such as Korean), I intend to only use languages which use the Roman alphabet. Unfortunately, this rules out some cases I'd like to use, but that's the technical limitation. The fact that I'm not using morphology in this experiment suggests that languages chosen should be primarily analytic; agglutination and fusion rely too much on morphology. Some possible languages to try: English, German, Spanish, Portuguese, Italian, Esperanto, Chinese (via Mandarin or Cantonese Pinyin or Jyutping), Trique, and Romanized Sindarin. In a couple cases there are several closely related languages, intended to test how well this thing can distinguish relatively small differences.

Orthography is pretty straightforward. Each letter, as well as several before and after it, are input into the orthography functions. I'm thinking two letters before and after, but we'll see. The output would then be whether the function thinks that letter fits with the letters around it.

Syntax is substantially harder. It's infeasible to look at words as atomic units, because there is no good way of representing them as such in our algorithms (especially neural networks). So, I'm kind of having to get creative (of course this is assuming I even have time to do syntax analysis). What I'm thinking at the moment is to look at one word as well as the two words immediately before and after it. Rather than trying to process the entirety of each word (which can't readily be done, due to representation problems), I was thinking of only using the first and last so-many letters (I'm thinking three) from each of the words.

This idea isn't as arbitrary as it sounds. For the languages I'll be dealing with, many words should be six characters long or less, meaning the entire word is considered. For larger words, where the center is not able to be considered, I rely on the fact that the beginning and end of words have been shown to receive more processing than the middle, and, consequently (in a positive-feedback-like manner), they tend to contain the most important information, such as indications of the word class, inflections, and derivation morphemes.

& More Awesomeness

I can't recall if I mentioned this before: while looking through the AI teacher's personal collection of AI journals, I ran across an article about the creation of an AI that could generate Chinese calligraphy. You give it a variety of examples of real Chinese calligraphy, it learns from those, and then by randomizing various parameters it's able to create novel calligraphy that is acceptable to expert Chinese calligraphers. Alternately, it can be used to create personalized handwriting fonts - you give it half a dozen characters in your own handwriting, then it generates the tens of thousands of other characters by mimicking your handwriting. There's one particularly impressive example where the AI generates the character "forever" (I think it's 永, but it's so artistic that it's hard to tell) that mimics a hand-drawn sketch of a horse.

The reason I bring this up now is that, while the article is controlled by the journals and sold for $20 a copy, I've discovered that a PDF of the article can be obtained for free on the site of an unspecified university in Hong Kong, which can be found with some looking on Google. The article is Automatic Generation of Artistic Chinese Calligraphy by Songhua Xu, Francis Lau, William Cheung, and Yunhe Pan.

Google Epic Fail

As the following indicates, Google sucks at Japanese translation.

"Work with everyone involved to meet the time that I talk directly to the image more than anything to receive a lot of opportunities, so those places will eventually work on the float's theme song has become melody It is disappointing that many of the Yes."

& Intellectual Property

I just finished writing a mammoth post in the discussion of this Ars Technica article about copyright and history. This more or less serves as my thesis on copyright, intellectual property, and file sharing in general:

After reading Alfonse's posts, I thought I should provide an alternate perspective/rebuttal from an intellectual property creator. I produce works of fiction, I produce works of information (the majority of my blog, for instance, dealing with topics related to programming, linguistics, and philosophy), and I produce works of computer programming (though this list is not all-inclusive). I spend just about all of my time creating in some way, even if what I create doesn't leave my head. Given that, most of my work is done primarily for two different reasons, which apply in varying degrees to different works:
1. Because I love what I create, and I enjoy creating it. So much so, in fact, that I do it even when I never expect to profit from it. If I didn't love and enjoy it, I wouldn't spend the considerable effort to create it.
2. Because what I create has value to others - it's useful in some way to people other than myself. Now, I'm still selfish in my own way - if I'm going to create something that I don't enjoy creating, even if it benefits others, I better be getting some other kind of incentive, such as money. But most of the stuff I create for the benefit of others is also enjoyable for me.

Conspicuously absent from that list is to make money. To me, getting paid for creating something is the icing on the cake - the cake itself is the content I produce. I think that, as much as is possible (hold that thought), that's the ideal of creation - not professions and commercialization.I believe that work created for its own merit is superior to work created solely for the purpose of obtaining your next paycheck; so yes, I think a world of amateurs would be great.

Of course, it's entirely possible that my beliefs are influenced by my own conditions - I have many skills, some of which can be used commercially, others not; so many, in fact, that I don't have time in my life to explore all of them (interestingly, writing is one of the less intellectual of my skills). If this is not the case for you, Alfonse, then that could certainly contribute to our difference of opinions.

Alternately (or perhaps additionally), perhaps our difference of perspective is due to different origins. When I read your posts, I can't get the image of a five year old throwing a temper tantrum when they can't get every single thing they want out of my head. I, on the other hand, have spent more than the last decade in an internet community based on free distribution of content. Content produced by others is freely used as the basis of derivation over thousands of hours of labor, and both derivative content and original content are given away without any expectation of (or request for) payment (some, such as myself, have even refused attempts by fans to pay for content that was intended to be free). In fact, I'm both a creator of derivative works and of original works.

On the other hand, creation of some things can be prohibitively expensive. You are absolutely correct in that big-budget movies (which I would love to see some of my stories made into, by the way), large computer programs, etc., would not be created unless the creators expected significant returns, because without those returns they would be impossible to produce. As well, some industries, such as the movie industry, cannot provide their work as a service, as you also pointed out.

Consequently, I'm (almost but not quite paradoxically) also a proponent of copyright (as well as patents, although that's an entirely different topic). Of course the copyright I'm a proponent of bears little resemblance to the copyright of today. I advocate copyright of 10-20 years, maybe 30 at most. Furthermore, there are going to have to be some changes in the nature of copyright, not just the duration, as well. This statement requires some explanation.

The world has changed. This is no longer the same world as when copyright law was established; it's not even the same world as twenty years ago. Personal computers and the internet have fundamentally changed the nature of information. I found Mark Twain's arguments in favor of copyright - specifically, that it's unfair for publishers to continue to make money off of his works when the copyright has expired and he no longer able to make money off of them - to be convincing... for the world at that time.

At the time, there was only one reasonable way to make a book: massive and expensive printing presses. As such things were well outside the reach of the common person, this gave publishers non-exclusive production rights even without enforcement of copyright. As they controlled the production, they also could set the prices people must pay for them. Furthermore, the books produced had an inherent value by nature of the cost of producing them. This was true even as recently as twenty years ago; recordable cassette tapes existed, but were inferior to the originals, and CDs could not yet be burned on a personal computer.

This is no longer the case. Personal computers and the internet have made it possible for a work to be duplicated an infinite number of times with absolutely 0 production costs (note that I'm referring to producing the copies themselves, not the content). Copies no longer have an inherent value, nor is there any limitation on who may produce these copies. Most computers now have CD and DVD burners, and information my be transferred over the internet without any special kind of hardware.

Twenty years ago, "theft" (I'm not even going to get into the debate of definition) of intellectual property was limited to those with large budgets - large-scale counterfeiter out for nothing more than profit - who could be reasonably attacked legally. Today, however, there are tens of millions (probably hundreds of millions if you count people outside the US) of IP "thieves", each wanting nothing more than to access the content themselves, with no exchange of money. Besides the fact that the fundamental nature of infringement has changed, it's quite literally impossible to target more than a small fraction of them with legal action, assuming they're even in a country you are on good terms with.

Under such conditions, copyright law enforcement will be, at best, sporadic, and at worst entirely unjust, as companies and governments are required to drastically lower accuracy of prosecution to be able to afford it at all (this is what the RIAA faces now, and it'll only get worse if personal infringement becomes criminal). Technological countermeasures (note that technology and computers are probably my #1 skill, and my current profession, so I hope you'll consider my words to carry some weight) are also doomed to failure without catastrophic side effects of the type that damage culture and invention more than piracy and counterfeiting ever could; though to be fair, such results would also occur if copyright continues to be strengthened as it has been over the last couple hundred years.

We are at the point where the morality of piracy is becoming irrelevant. Regardless of whether it's right or wrong, it's simply not possible to stop it, so you're going to have to get used to it. Even better, you could follow the lead of me and my community and thrive on it, both commercially and non-commercially. I have a job; how about you?

Random Picture of the Day

Apparently they poke massively fat cats in Japan, as well.

Random Fact of the Day

The Japanese particle 'wa' (は) it typically called the topic particle. It indicates a topic, and then subsequent sentences refer to that topic in some way (the topic may be the subject, object, or some other case in the following sentences).

However, that is not the true meaning of the particle. What it actually is is an emphasis particle - it emphasizes some word or phrase. It's only by convention that it's come to be used to indicate the topic, and at times it's also used for other things.

So really, it's like "War! What is it good for?"

Conjecture & Reality

A bit ago I posted some of my own hypotheses about SBC/Yahoo! DSL BitTorrent behavior, and how their "network management" system (as Comcast likes to call it) might work. Now, in the FCC's current investigation of Comcast's network management, Comcast has finally given (some) details of how their system actually works.

The actual filing is available, and Ars Technica has a brief summary and discussion of the filing (I haven't read most of the filing myself, yet; only the Ars Technica commentary).

& Artificial Intelligence Class

So, the spring semester has begun. In fact, the spring semester is about 22% over already. It looks like I'm going to have to do term paper and/or project in each of my four classes - advanced game programming, graphics programming, artificial intelligence, and software engineering. This post deals specifically with the prospect of the AI term project. On the first day of class, I'd already come up with two abstract ideas to do (ideas for topics, but not specific projects), and two concrete ones, of varying sizes. I'll mention some about each of them. All of them have basic requirements of at least practical use, smartass value, or fun coding them (some have more than one of these).

The most practical project would be something related to E Terra. This was one of the general ideas that I didn't have a specific project in mind. The E Terra macro-AI (e.g. computer players) is going to be an absolute bitch to write, given the fundamental differences between E Terra and other strategy games. In terms of search space, it's like comparing Go to Checkers - there are simply too many possibilities to consider all of them, given the broad control of the race E Terra permits. Long term plans need to be developed, and then modified in reaction to actions of other players. Much more attention must be paid to what the enemy is doing, because the time it takes to counter something novel another player does is much longer than in traditional real-time strategy games; there's no bouncing back after the reaver drop, so to speak.

However, while the macro-AI is by far the most difficult, there are lots of micro-AIs that need to be written, as well - small (and relatively easy) self-contained tasks that form the building blocks of play. While these are more manageable, the question remains as to whether there's a single micro-AI that is complex enough to be used as a term project. Some specific micro-AIs that will be required:
- Pathfinding. Units must be able to find the most desirable path from one place to another, based on the knowledge of what's in between those two points.
- Threat assessment. A unit or a group of units has just discovered a group of enemy units. Can it overpower the enemy group? Can it outrun the group? Should it attack the group? Should it run away to safety (e.g. its nest, where presumably many other units will be)? Can it/should it try to lure the enemy group to a stronger group of allies and then fight?
- Combat. Selection of special abilities to use and when.
- Idling. Should the unit eat some food while it's busy doing nothing? Wander around randomly? Go to sleep?

Next is something related to linguistics, a member of the 'fun' category. Plenty of things to choose from, though specific ideas that are manageable aren't readily apparent. Unfortunately, it's unlikely that I'd be able to do something with practical use alone, in an undergraduate course, in the time permitted.

Next is something I'm somewhat attracted to of non-negligible practical value as well as significant smartass value: a simple file format cracker. A program where you give it several different examples of some file format, and it determines the file format structure as best it can. Obviously there would be significant limits to what any such AI could accomplish without true intelligence; e.g. it might be able to tell you the data types of values as well as what they seem to be (flags, file offsets, sizes, counts, "random" data, etc.), but it wouldn't be able to tell you the meaning of most values; similarly, it would have absolutely no way to crack encrypted or compressed structures (though, for that matter, even a skilled reverse-engineer wouldn't be able to do that, either, if they only had a hex editor).

This would be based on my assessment of my own thought processes while reverse-engineering file formats (especially the Mega Man Anniversary Edition and Guild Wars archive formats, which I reverse-engineered using nothing but a hex editor). Most of the stuff is pretty basic: looking for telltale clues about the location and format of fields, determining the stride of arrays, and then using statistical analysis and heuristics to guess what individual fields are.

Finally, something with extreme smartass value, which I think would also be highly amusing to develop: a dating/relationship simulator. While only the most extreme of nerds play such games outside Japan (and, in fact, most people outside Japan have never even heard of them), this genre of video game is spectacularly popular in Japan (possibly the most popular). Some examples of anime that are based on dating sim games (as far as I know): Clannad (currently airing), ef ~ A Tale of Memories (download), and Kanon (download).

I've considered writing a story made for a dating sim before, for the reason that I bet I could do irreparable harm to the genre. Not to mention the fact that it's been done so many times that it provides a real challenge for a writer (along with the school comedy genre of anime, which I'm also considering trying my hand at). Of course such a project here would focus on the AI portion - the simulation of emotions and thought processes of love interests - rather than any type of specific storyline.

Public Service Announcement

It looks like the rumors about Geneon USA going out of business are true. Geneon USA is, among other things, an importer/translator of anime and music CDs from Japan. It also happens to be the only label that has (legally) brought any of the work of Yuki Kajiura, one of my favorite composers, to America (although as I prefer to pay the authors for music I like, I wish they'd brought more of it...).

Thus, Geneon going out of business means that these soundtracks will not be available anymore, which is bad. I already see at least two soundtracks that are no longer carried by Amazon.com, and several others that are down to 5 or less in stock. So, if you're looking to buy some good music, now is your last chance.

Here's a list of the Kajiura CDs by Geneon, ranked roughly by how good they are, with links to them on Amazon (if Amazon is out of them, you might be able to find some from other stores; but you're on your own for that - just make sure the ISBN number matches these here), along with some samples:

Madlax OST 1
Nowhere
The Story Begins
To Find Your Flower

Madlax OST 2
Fall on You
People Are People
Bank on Me

Noir OST 1
Romance
Salva Nos

Noir OST 2
Le Grand Retour
Indio

Elemental Gelade
Beauty and the Beast
The Bond - Reverie Metherlence
Wings

Fiction
Fiction
Vanity

Le Portrait de Petite Cossette
Moonflower
In a Beautiful Morning of May
Undertow

Aquarian Age: Sign for Evolution

Mother Load!

So, while procrastinating harder than usual today, I happened across something very interesting (at least to me). While my grandpa hasn't written much about the Trique language in general (his papers have been more about specific aspects of Trique, especially those relating to his primary field of study - analysis of discourse), I've finally managed to find someone who has. All things considered, it shouldn't be a surprise that my grandpa and she are well acquainted, and have collaborated a great deal in the past; though that's somewhat beside the point.

The bottom line is that she has written a number of articles (scroll down to the On Trique section) and other things about Trique grammar (although hers is Copala Trique - a sibling of Chicahuaxtla Trique, which my grandpa studied and my Trique Bible is in). Most notably, she has written an entire manual on the grammar of Copala, which I'm just starting to read, and a Copala Trique dictionary. A number of her works are available online, including the manual and dictionary themselves.

Unfortunately, almost all of her works available for download are in Spanish. This ended up providing a surprisingly small obstacle for me, however. When I tried to read the grammar manual, I found that I was able to make out the Spanish quite well. Between the Spanish I remember and the fact that much of it is linguistic jargon (making it fairly easy to deduce the meaning of unknown words from context), I've been able to understand almost all of it that I've looked at so far. It would be incredibly ironic if I ended up buying one language and getting a second one free.

So far, I can see a few differences between the two. Copala Trique seems to like to delete some consonants that Chicahuaxtla Trique has, has fewer vowels, and I've heard Chicahuaxtla Trique has the most complex tone system of any Trique language. I also readily noticed some pronouns and number markers don't appear to be related between the two languages. It also seems like Copala Trique has gender-specific third-person pronouns that Chicahuaxtla Trique lacks, although I need to do more study to make sure (I haven't seen any that appear to be gender-specific in this Bible, but I can hardly be considered an expert on the topic).

In other tangentially related news, exactly one week ago somebody by the moniker G.broadwell started to add some limited details about phonology, morphology, and syntax to the Wikipedia article on Trique. As well, this blog is #5 on Google for "Trique", #4 for "Trique grammar". Finally, it appears that the Trique indians now have a web site (Triqui is a well-known alternate spelling), although it's in Spanish; isn't technology awesome?

On the Name of Q

Last summer, I started a new character in World of Warcraft. As the character I'd played before that was a Draenei (the new Alliance race added in Burning Crusade), I decided to make a Blood Elf (the new Horde race in Burning Crusade). A mage, to be specific, as killing things by the dozen is awesome.

I generally put some care into creation of characters in such games. Sometimes I'll create characters derived from my stories. And even if there is no character in my stories appropriate for a character class, I usually try to come up with an appropriate name and some coherent theme for the character. Sometimes these original characters even find their way into my stories... (one even became the main character of a series)

So, after I'd come up with a suitable theme for the character, I had to come up with an appropriate name. While I usually just pick names for how they sound, in this case I decided to modify some of the first thoughts I had (based purely on sound) so as to give it an appropriate meaning.

The name I ultimately decided on was Pyruvega Narovire. The first name was originally based on sound, and is a mish-mash of languages. 'pyru' is derived from the Greek word for 'fire', and 'vega' means, roughly, evil sorcerer. After getting the most important part of the name (the first name) done, I felt a bit more free with the last name, and decided to use Quenya based on meaning (though the partial rhyming of the two names was not a coincidence). Here, 'naro' means 'of flame', and 'vire' means 'rose'.

So, fast forward a few months. After I got back from Kansas after last summer, I went to download some new anime series. A couple of them that I wanted were only available through IRC distribution bots. As I hadn't used IRC in a couple years, I really didn't have any nickname that I was using. For some reason I decided that I would translate 'Pyruvega Narovire' into Japanese.

For those not familiar with Japanese, the Japanese language is something of a mess. It's a mixture of true Japanese words and many words that were borrowed from Chinese (and more recently English, though English words mostly relate to technology). Worse, the Japanese usually learn both the Japanese pronunciation(s) as well as the Chinese pronunciation(s) for words present in both languages. The reason for this is that there are many cases where the Chinese pronunciation of a Japanese word is used; one example of this is in names.

Names in Japanese consist of some pronunciation encoded as a series of characters (usually kanji borrowed from Chinese, but sometimes Japanese hiragana). In names, all pronunciations of the words/kanji are fair game, and it's not unusual to see some kanji of the same name pronounced as in Japanese, while others are pronounced as in Chinese (from what I've seen, Japanese first names have an average of two kanji). This allows Japanese names to be very poetic, as there are often a large number of ways to write a given name, each with different kanji (thus meaning different things). Even worse for Japanese-as-a-second-language people, there are many ways to pronounce a set of kanji, and often only tradition will tell you which is correct (though if it's a name I made, tradition won't help you, either).

So, I had two tasks: to pick a set of kanji for my name, and to pick a pronunciation for that name. The kanji were relatively easy to figure out, though I also did some picking of pronunciation at the same time. The kanji I chose were 炎 (flame) 魔 (witch) 火 (fire) 野薔薇 (wild rose).

As the first name was a mish-mash of languages unrelated to Quenya (the "true" language, we could assume), I thought it would be best to use Chinese ("foreign" words in Japnaese) pronunciations for the kanji in it; this made the first name Enma (炎魔). As the last name was pure Quenya, I decided to use pure Japanese, making the name Hinobara (火野薔薇). Of course, the Japanese (as well as many other east Asian societies) write their family name before their given name, but never mind that.

For trivia value, both Enma (many different spellings) and Hinobara (日野原) are real Japanese names, though they are both family names, and they use different kanji than I used. Enma is, however, also the way the name Emma is transcribed into katakana (エンマ).

On a tangentially related note, one misspelling I accidentally used at one point ('Inma Hinobara') is an anagram for "I am a ban rhino", which was awesome.