Reinstalling Ubuntu

One thing that really frightens me is reinstalling an OS and keeping the "lights on" with regards to my personal data and extra apps.  For this reason, I have never reinstalled the OS on my main workstation.  In fact, when I upgraded from an old dell laptop to a desktop machine, I only moved the HDD over and ran it as is.  The original install was 32 bit and was at least 7 years old.

Not that I don't upgrade.  It started as plain Ubunutu.  It didn't take long before I decided I was a KDE person and I "converted" it to Kubuntu.  At some point the new KDE came out and I decided to try unity since by that time it was already part of the upgrade.  It didn't take long to throw that out and "convert" my dist into Xubuntu.  I've been happy with Xfce but I could tell there was a lot of cruft running around from the previous desktops.

Enter windows 8.  Back when I bought my new desktop, it came preinstalled with win8 and had UEFI enabled.  Since I was reusing my old install, it required a legacy bios and I was relieved to find that the PC bios supported legacy mode.  However, this meant that windows 8 was no longer bootable.  I could live with that though.  Until I wanted to install a game for windows on my PC.

After doing a little googling, it appeared that grub now supported UEFI and I found this nifty program boot-repair which aims to fix things properly.  So I tried to flip the switch to UEFI and try boot-repair only to be told that my current linux install was not compatible with UEFI.  I believe it was because it was 32 bit.   This put me over the edge to do the upgrade to a proper Xubuntu 64 bit install since it is not possible to upgrade from 32 to 64 bit in place.

I made to sure to tell the installer to reuse the existing partition.  This left my home dir intact.  Some of my old /usr/local installs no longer worked, and I had to remember or discover which extra packages I had installed (digikam and kmymoney for example).  But that was mostly quickly solved.  The only rub was that it seemed at some point my gpg public key ring was replaced by an empty one.  My secret ring was still in place, but gpg won't run without the public key.

I am still not sure what happened, but thankfully I did back up my home dir to the windows partition before installing Xubuntu and was able to copy my public ring back and all looks good.

Hopefully this is the last I have to write about this experience.  It went surprisingly well - even to the point where it seemed less painful than an upgrade.

Trapped between a verizon and a comcast

I have no love for either provider.  My verizon bill went way up unexpectedly this month which triggered my "switch to the other guy" clause.  Here is my chat with verizon about trying to fix my cost.

Live Chat Transcript

It was a pleasure to assist you. Your chat session has ended. Thank You. 

Your chat ID number is 11161383280.

Please select Exit to close this window.

Print

Exit

Chat Subject:Upgrade Existing Services

Your Question:Convert to digital voice

A Verizon Service Representative will be with you shortly. Thank you.

09:28:47 We apologize for the delay.You are next in the queue. A representative will be with you shortly.

Agent Kim has joined. (09:28:47)

Kim : Chat ID for this session is 11161383280. (09:28:47)

Kim(09:28:57): Thank you for choosing Verizon and visiting our chat service. I would be happy to assist you with your question.

For quality and security purposes, your session is recorded and may be monitored or reviewed. Please do not provide sensitive information such as social security, bank account or credit card numbers to the chat agent. May we view your selections and/or account information, including any services you subscribe to, which are displayed to you on verizon.com, so that we may assist you with respect to available Verizon products and services? Under federal law, it is your right and our duty to protect your account information. May I have your permission?

ANDREW BARBER(09:29:15): yes

Kim(09:30:31): Hi Andrew! Thank you for being a valued Verizon customer. I will be glad to assist you.

Kim(09:31:06): To be sure, would you like to change your home phone to Digital Voice?

ANDREW BARBER(09:31:16): I am thinking about it, yes.

Kim(09:33:06): Great! Let me first help you with the link so that I can help you with the steps to make the changes now online.

Kim(09:33:46):http://www.verizon.com/ForYourHome
Please click the above link and let me know once the next page is up.

ANDREW BARBER(09:34:50): page is up

ANDREW BARBER(09:35:26): don't see any bundles I like

Kim(09:35:56): Perfect! Could you please share what all plans you currently have?

ANDREW BARBER(09:37:39): I have 15/5 internet and regional value calling

ANDREW BARBER(09:37:50): But you should know that already

Kim(09:38:20): Thank you for sharing the information.

Kim(09:38:35): Now let me help you with the steps to check the Digital Voice plan.

Kim(09:39:00): Are you able to see "Create your own bundle" section on your account page?

ANDREW BARBER(09:39:07): yes

ANDREW BARBER(09:40:19): Let me short-cut this process. I can go to comcast and get internet and digital voice for 39.99 per month. What do you have to offer? I only want internet and digital voice.

Kim(09:41:39): As you already have the service so let me help you with the steps to check the price.

Please allow me to help you with the steps.

Kim(09:42:29): Under "Create your own bundle" section, please uncheck TV and select the internet plan and Digital Voice Unlimited.

Kim(09:42:40): Please let me know what price you are able to see for your account.

ANDREW BARBER(09:43:31): 79.99

Kim(09:46:26): Perfect! You would be able to get FiOS Internet and home phone for $79.99/mo.

The other service providers are does not offer 100% fiber optics straight to your home.

ANDREW BARBER(09:46:38): That doesn't mean anything

ANDREW BARBER(09:47:00): I've experienced both and I can tell you there is no noticeable difference

ANDREW BARBER(09:47:16): So, that is not worth $480 per year to me.

Kim(09:48:01): I do understand your concern for price; however, this is the best price you would be able to get with the FiOS service for the bundle.

ANDREW BARBER(09:48:34): Can you transfer me to customer retention department?

Kim(09:49:55): Sure! I'll be able to transfer your chat; however, the online price will be same and you will get the same price for the bundle.

ANDREW BARBER(09:50:46): Thank you.

Kim(09:51:26): Would you like me to transfer your chat?

ANDREW BARBER(09:51:33): Yes please.

Kim(09:52:18): Please hold for a moment while I transfer yout chat.

This session is being transferred. (09:52:28)

This session is transferred to Lynette. (09:52:43)

Agent Lynette has joined. (09:52:43)

Lynette : Chat ID for this session is 11161383280. (09:52:43)

Lynette(09:53:03): Hello. Thank you for choosing Verizon and visiting our chat service. I will be happy to help you today.

For quality & security purposes, your session is recorded and may be monitored or reviewed. Please do not provide sensitive information such as social security, bank account or credit card numbers. May we view your account information, including the services you subscribe to, so that we may assist you during this chat with respect to available Verizon products and services? You may deny us permission, which will have no effect on your current services. Under federal law, it is your right and our duty to protect your account information.

May I have permission to review your account?

ANDREW BARBER(09:53:53): yes

Lynette(09:55:28): The offers provided by Verizon are shown online and are unable to be changed. If you would like to speak to someone in regards to canceling your services, you may call in directly to (800) 837-4966 and they will be able to place the order for you at that time.

Lynette(09:55:33): Thank you for choosing Verizon, we appreciate your business. The department to which we have referred you will be able to assist you. If you have any additional questions, please do not hesitate to contact us again.

Your session is now closed.

Welcome to Verizon Live Chat

Print

Copy to Clipboard

Privacy Policy | © 2013 Verizon

Misterhouse notes

Just some notes for myself about recent learnings around misterhouse and insteon.

Device Linking

From what I understand now, when adding a new device to my network, I need to link it with the PLM.  To respond to the PLM (and by proxy MH) , I generally don't have to do anything but set up the device in the MHT file.  Newer devices this year, however, have changed the protocol, and MH can longer automatically discover and link the devices into the PLM as a responder to the PLM.  So, for newer devices, I have to manually link by using the set button on both the device and the PLM.  

For Linc devices, linking is a two way street since the device can send status to MH when the button is pressed.  For this, I generally have to hold the device's set button for 3 seconds, and then in MH under the PLM control, select "complete linking as responder".    It may vary by device on how to put it into link mode as a controller (ie update the PLM when the button is pressed).

Email troubles

Today I received an email from the guy that runs the server I use as a mail relay.  Apparently, my asterisk box has been sending constant email to his mail server (smart host forwarding).  I think what was happening is that I had set my .forward on the asterisk box to send to "user@host" where host is my own server at home.  That server uses exim4.  I really could never figure out how to make "user@host" stay local.  Via some trial and error I finally found that "user@mydomain" seems to work and the email isn't forwarded to the smart host but stays put on my server.  I just need to remember this in the future, hence my post here to remind me.

UPDATE:
I didn't have it quite right before.  A couple of issues.  First, the asterisk box wasn't delivering email to local users.  Instead it punted and sent it to the relay which didn't recognize it either.  This was fixed by setting mydestination to "localhost, <myhostname>" in the postfix config file.  After that, mail would get delivered locally when using unqualified user names for the email address, as well as user@<myhostname>.  The second issue was that my exim4 main mail server didn't recognize user@hostname (where hostname is the exim4 server name) as being local and would send it to the relay.  This is the same problem the asterisk server had, but this is fixed in update-exim4.conf.conf by setting dc_other_hostnames to include the exim4 server name.  It previously only had my domain name in it which is why user@<mydomain> was getting delivered locally.

Typical

I hate it when this happens but it happens far too often with upgrades.  I upgraded my laptop to Ubuntu 12.04 (which is another post to be made).  About a month later (today) I need to put some photos from my camera into digikam.  When I start up digikam I get this error:


You have insufficient privileges on the database.
Following privileges are not assigned to you:
CREATE TRIGGER
Check your privileges on the database and restart digiKam.

A google search turns up https://bugs.kde.org/show_bug.cgi?id=262321 ( a Jan 2011 bug)

Basically, some change was made to digikam in how it uses the database, but it doesn't quite work right with mysql.  The work around is to give the digikam user full privs on the entire mysql database.

There is this comment:

--- Comment #15 from davidebasilio <marsicanbear gmail com>  2011-11-07 16:12:55 --- 
Digikam 2.3.0 is out http://www.digikam.org/drupal/node/632
But it looks like noone took care of this bug since 2.0.0 
Will it ever be fixed? 
BTW, this is tagged as wishlist, but I do not really think that's correct. This 
is a bug. 


</end comment>


So, it's almost 18 months later and this is a still an issue!  The glacial pace at which these fixes get introduced combined with the super fast pace new features (and bugs) get introduced really frustrates me.  Now I have to do a work around which some years later will cause some problem and I'll never make the connection.


Check out this comment from the bug:

Bruno Friedmann 2011-03-31 08:13:20 UTC

Marcel, in fact I don't know how you can do that, due to upstream (bug/feature) the creation of triggers apparently are root reserved in mysql, due to replication feature of the database engine. ref http://dev.mysql.com/doc/refman/5.0/en/stored-programs-logging.html So for the "normal" embedded mysql there's no troubles as it's run like superuser by end user. The real impact is about using digikam in shared mode (centralized mysql server). So digikam can said : hey we didn't support that, but as it has, it will become a big regression. Time to interface digikam to virtuozzo ? :D

I mean, why do they offer the option to run shared mode if they aren't going to make sure it works on every release?  This is one of my big gripes about FOSS software - they build lots and lots of doors that you can open, but only a few will lead you to safety and many lead you to your doom (although they appear perfectly safe).  I chose the door to using a shared mysql server and I am now paying the price.




UPDATE (6/25)--
I finally got digikam working again.  There were numerous issues and I am not sure which one was the culprit.  First, some operations in mysql gave me errors or warnings about some mysql internal tables that had the wrong number of fields (20 vs 16).  It seemed this was due to an upgrade that didn't complete normally.  I ran mysql_upgrade --force and complained about some ampache tables that needed repairing.  After that it gave me this error:

Error: Incorrect database name '#mysql50#mysql.orig'

I had to remove the mysql.orig directory from /var/lib/mysql

 After fixing those, it seemed to update the database correctly.  The other thing I did (and maybe this wasn't required) was add the
create_index_if_not_exists() proc to the digikam databases per this bug (https://bugs.kde.org/show_bug.cgi?id=276052).

Now digikam works again.  But if you search for "digikam failed to upgrade" you will see lots of threads talking about failures to upgrade the schema for digikam from 5 to 6.

Ampache woes

So at some point something went horribly wrong with my ampache setup. I decided to try the ampache script that renamed files using the tag information. This seemed like a good idea, because my files weren't all named or organized in a consistent way. After a couple attempts, I noticed that all my songs in ampache were by an artist who's name started with the directory path to the file. Great. At the time I was pretty sure I had messed up the mp3 tag data because after rebuilding the catalog in ampache, nothing changed. Artist name was still the directory name (/mnt/usb_mmedia/music/itunes/...). I tried to fix things a couple times but I seemed to make it worse.

Fast forward maybe a year to the past week. I decide it's time to go fix my library. I wanted to start by fixing the MP3 tags, so I downloaded some utils and started to work. But I couldn't find anything wrong with the mp3 tags. The artist was correct here. I also noticed when playing a song through ampache that the artist name seemed to come out correctly there as well. So what the heck is going on?

I start putting print statements in the ampache code to see what it is doing when I verify the catalog. I eventually discover that it is not looking up mp3 tags when building the catalog and instead trying to generate the information from the filename. This is happening because I no longer had an entry in the ampache config file for mp3 tag processing order. I put that back in and everything goes back to normal in Apache.

So why was I missing that config line? It goes back to another problem I had with ampache where catalog verification/creation would hang at some point as the mp3id library would get stuck and chew up memory. This was a known issue with ampache which happened on some files who have corrupted tag information. It was quite hard at the time to identify which files these were and besides, I didn't know how to fix them. The only option was to stop ampache from reading the mp3 tag information by removing the tag order config key. It worked find until I tried to verify the catalog.

This is exactly why I do not like to employ hack work arounds, they always come back to bite you later.

The other stupid thing about all this is that ampache was using a deprecated php method (eregi) and would spit out errors during catalog verification. The response from the ampache devleoper was to downgrade php to an older version or use the alpha version of ampache. Really? What kind of choice is that? Have you ever tried to downgrade something in linux? Remember these systems are a tight-nit web of dependencies. You can't just go around back-revving major packages. I don't know how the stable version of ampache was even going to work on recent distribution releases. I had to replace all the calls in the ampache source code with the proper methods, then everything worked. Thanks open source!

Long time, no update again

Well, I am still on Ubuntu 10.10. I plan on using LTS releases only, but even then the support isn't there. Case and point:

https://bugs.launchpad.net/ubuntu/+source/indicator-application/+bug/569273

So, my gnome-power-manager looks like this:
Size: 443168 kB
Rss: 411364 kB
Pss: 411364 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 72 kB
Private_Dirty: 411292 kB
Referenced: 309920 kB
Swap: 31704 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB


Seems pretty excessive.

This is ridiculous. I would think that the gnome-power-manager is a fairly important piece of the OS here. Why in 18 months hasn't this been fixed in Ubuntu? OSS support sucks. Sure, make the argument that paid software isn't much better, but why would OSS hold itself to that shitty bar? When is Ubuntu going to be a robust complete product? Never. That's how OSS works. Constantly changing and never stable. Even the stable stuff isn't a safe bet because if you find a bug, odds are they've already moved on to a newer, better version with different bugs for you to discover. Nobody fixes bugs on old releases. Not even Ubuntu, even when it's in the support window.

Ethernet device renaming

At some point during the last five years of upgrades on my laptop, the ethernet port went from being eth0 to being called eth2. I couldn't for the life of me figure out how or why this happened. I also couldn't figure out how to undo it. Well, during my recent pogo issues, I finally ran into some web pages which told me all about it. In a nutshell, it is udev. Here is the file that can be modified to change the device name of your ethernet devices.

/etc/udev/rules.d/70-persistent-net.rules
# PCI device 0x14e4:0x170c (b44)
SUBSYSTEM=="net", DRIVERS=="?*", ATTR{address}=="00:15:c5:3c:e6:4d", ATTR{type}=="1", NAME="eth2"

What happened to me was that eth0 and eth1 were assigned to MAC addresses based on my previous laptop. Dell had to replace my laptop and when that happened the new MAC addresses had to be given new names as eth0 and eth1 were already taken from my old laptop.

Pogo down

Currently battling an issue with my pogoplug. Unfortunately it provides everything like email, home automation, database etc. It seemed two nights ago that it just stopped responding to network traffic. ping worked, but ssh and web were refused. So I power cycled it, but the result was the same. I could hear the disk booting the OS, but this time pings to the machine wouldn't even work. ARP showed that the it hadn't even asked for DHCP (no arp mapping).

Concerned about my HW, I rebooted into the original pogo linux and the network came up fine. I tried an fsck on my root drive and it showed no issues and still no network. I even tried another pogoplug without success. I think either the kernel is corrupt or something is running out of order in the init process such that it hangs before bringing up the network.

This morning I installed a fresh debian onto a usb key and it booted. This evening I will attempt to determine the difference between this fresh install and my broken one.

Update: I have mostly recovered my system. I am still not sure what the issue is. If I kept ping running to that machine while it was booting it would consistently fail, but then come alive for 4-5 pings and then go offline again. The last message I could find in dmesg (when mounting the drive from another machine) was a normal message from the NFS kernel module.

I tried disabling various services in startup to try and narrow it down (including nfs and misterhouse). At one point my system came alive but I went in and restored some of the init and it wouldn't boot to network again. I tried setting my root disk to mount errors=continue instead of errors=remount-ro thinking that it was remounting root ro and making it impossible to write any run time files or update devices. That fix seemed to help, but subsequent boots were hit and miss.

Finally, I commented out all the local usb drives I had in fstab. I just left root and the home dir disk. Magically the system became stable. I have no idea how or why this was causing a problem with the network on boot. None of those disks were network mounted. I guess I'll be reading more on mount and fstab to see if I can explain this. In the meantime my email, home automation, and vbox images are back online.

UPDATE: 3/24/2011
I managed to restore fstab to automount all the usb drives. I had to remove the "auto" fs type to make it work. My pogo now reboots without network issues. I could have sworn that it worked for a while with the auto fs in place, but maybe not.

No news is good news?

Well, it won't be long lived. Recently I started having memory leak issues in the OS. After about a day, all the memory would be used up leaving an unusable system. It started with a power manager, I can't remember if was the gnome one or not. But after kill -HUP on that process, another process called polkitd would start leaking memory. I have to HUP polkitd once a day now. Some googling showed that there are some instances going back to Lucid (I am on Maverick right now). I can't believe that something as core as polkitd would go this long unfixed. (Apparently polkitd is some sort of policy daemon, security or power I am not sure).

On another thread, I've been having VOIP issues. Whenever Rosalia calls me at work, the line is frequently choppy which usually means UDP packets are getting lost or delayed. I found this great article on how to configure Tomato QOS which finally explained to me how to use Rate and Limit properly.
http://www.linksysinfo.org/forums/showthread.php?t=60304

Rate is actually the min guaranteed bandwidth for a class. So now I make sure my rate percentages all add up to 100%. Calls since this fix appear to be much better, but I still can detect a few dropouts here and there.

KDE again

When Ubuntu went to KDE4, I quit. That is I quit being a KDE user. Initially, I found it to be much more interesting than Gnome, but Ubuntu's hasty jump to KDE4 sealed its fate with me. By that time, Gnome was in pretty good shape and I found it to be more stable and more intuitive. I still liked certain KDE apps though, like Amarok (before the screwed it up) so I kept the kde libraries on my system when I converted to Gnome.

Fast forward a couple years. I now use Kmymoney and Digikam. They are great apps and do what I need. At least until I upgraded ubuntu from Lucid to Maverick. Now all my KDE apps suffer from insufferable lag. I first noticed it with Kmymoney. It would take about 3-4 seconds every time I committed a new transaction. Previously it took under a second. Now I am finding that Digikam is pretty much unusable. It takes 5-12 seconds for it to respond to the delete button or clicking on a different photo. Makes it kind of hard to tag and prune any pictures I take. I don't know what's wrong as there isn't much explanation I could find via google. And this comes right at Christmas time when I am taking a ton of pictures.

This is exactly why upgrades scare me. Unfortunately, I am always forced to upgrade to get fixes for other issues. Linux, can't live without can't live with it. I feel more and more that it is not an OS that I can depend on.

Ethernet woes

So an old problem has recently revisited me. The b44 kernel module which provides the driver for my broadcom ethernet chip has issues. Mainly it likes to reset under heavy traffic. You wouldn't normally notice except that when using vpnclient it disconnects the VPN connection which is very annoying. I did some searching around previously and found some commentary by the driver maintainer that this resetting behavior was on purpose and they had no plans to change it. So I went to ndiswrapper and after some futzing around I got this to work. So now it doesn't disrupt my vpn session, but apparently it causes the vpn kernel module to sometimes crash and hang parts of my system after I disconnect the vpn :( This is what I call typical linux behavior. I am constantly trading one bug for another. Sometimes it's my choice, sometimes not :)

Now, if I can just figure out how to stop the system from loading b44 at boot. blacklisting it didn't seem to stop it.

Here we go again

Last night was another exercise in frustration. I decided to try skype again so I called up the parents and waited for them to log on. I call them and get audio and video from their side, but they can't hear or see us. No LED on the webcam confirms it's not working. I try "rebooting" the camera by unplugging it and plugging it back in again. Another call and video is working! However, audio is still not working. I go into the sound manager (Gnome) and see that the webcam is not selected for input (can't the system mix different inputs?). I select it and see the VU meter working but still no sound in skype. Restarting skype and everything works. I don't understand why pulse or whatever isn't making the microphone available to skype dynamically.

Here's a nag from the archives - my video woes on my E1705 laptop. It has an ATI x1400. Short story - I've never had reliable 3d from this card. Long story: In the beginning I had to use the ATI catalyst drivers and like everything else in linux it was a trade off. I had to trade sleep/hibernate for opengl. But no matter what I did, I couldn't get Beryl or Compiz to work. Eventually, ATI stopped supporting the X1400 in their driver updates and it still had issues with suspend (there were probably some other issues as well). By that time the open source driver supposedly had 3d support. I found that some things seemed to work, but googleearth would crash after a minute or two. So I've never got full use out of that video card, except of course when I actually ran windows :(

In the beginning

Well, I decided that it was time to start documenting my many frustrations with linux. This is going to be a venting space for me. I hope it helps relieve the anger that I sometimes feel. I should be clear in stating that I use no other operating system at home besides linux. I flirted for a few years with OSX, but found that I couldn't upgrade the OS with out upgrading the hardware. As far as linux goes, I have been using it at home since 1996 or so. Perhaps I know too much.

I was originally a Red Hat user, but I became really frustrated with the packaging system. You had to "discover" dependencies one layer at a time as you installed. I kept seeing this thing called APT being referenced on Slashdot and finally gave Debian a try. I really haven't looked back since, although I run mostly Ubuntu now.

Tonight's frustration is with Skype (again). Last week I rediscovered the magic environment variable I needed to set before running Skype to have video actually display. Turns out that running the GL version of Cairo-dock prevented video overlay from working (unless you set the magic environment variable). That's just great. I should note, this is a prime example of linux "just not working". I had originally run into that problem trying to get mythtv frontend to display video, but had forgotten all about it by the time I tried running Skype.

But I digress. I'll sum up past frustrations in a future post, but tonight's issue is with sound. It started fine, but then audio pretty much stopped. After killing skype and trying to edit sound settings, the sound settings tool gave me "waiting for sound system to respond", which it ultimately fails to do. Seems that pulse has gone out to lunch. Restarting pulse didn't fix the issue. Seems I am faced with Yet Another ReBoot (YARB). I might get away with restarting, but I haven't tried anything yet. I decided to start this blog first.

This is the latest in a long series of frustrations with Skype and linux on this computer (Dell zinoHD , ubuntu 10.04) and my laptop (Dell E1705). I swear, every time I use skype to talk with my parents, something goes wrong. My dad is really not impressed. His windows box never has a problem. Why is this so hard with linux?

Time to reboot. I'm sure I'll be back within the week.

update: unplugging the webcam (logitech c260, uvc driver) then plugging it back in had some success. Audio output started to work, but then the microphone on the webcam would not show up in the sound preferences. Is there some magic to make this happen? I guess so - I had to unplug and plug it back it twice more after which the microphone started working. Skype lasted about 10 minutes before sound input and output ceased to work again. *sigh*