2017

Coaching VTA Bus Drivers Again

This morning I stood at the bus stop with the usual suspects waiting for the pre-dawn express bus to Palo Alto. It was five minutes late. We watched in amazement as the express bus drove past us across the street, coming from the opposite direction for the afternoon route. Before the bus could finish turning right on to Southwest Expressway to enter the loop-de-loop at Meridian Avenue and the 280, someone had the VTA customer service number on speed dial and started talking to a customer rep. We shook our heads. This isn’t the first time we had to coach the bus drivers on their new routes.

The Valley Transit Authority (VTA) in Silicon Valley rotates the drivers among the different routes every March and September. This week, the second full week of April and six weeks after the last rotation, some schedules got tweaked and new drivers started on the routes to my tech job in Palo Alto.

The local bus I take to pick up the express bus arrived two minutes earlier on the revised schedule, which meant I had to get out of my apartment five minutes earlier. Some drivers will arrive a few minutes earlier to avoid getting stuck behind the lights at the light rail crossing or take a break at the 7-11 at the next stop. New drivers on new routes are an unpredictable bunch, especially when sticking to the schedule.

The express bus tried to pick us again. Now ten minutes late. The driver probably had to go back out on southbound 280, swung up and around on Bird Avenue, and came back on northbound 280.  Still coming from the opposite direction by taking the Meridian Avenue exit instead of continuing on to Southwest Expressway. The onboard GPS should have given the driver precise directions for getting to this particular bus stop. If the driver is early or late, the GPS gives them a notification. The GPS either not worked or this driver ignored it.

The person on call to customer service got patched through to the driver over the radio. “You stay on that side,” he told the driver, pointing at the other side of the street. “We’re crossing over.”

A half-dozen of us ran across the empty lanes like East Berliners trying to cross the kill zone to West Berlin. The express bus veered across the lanes as if the driver was going to pick us up in the middle of the street, make a U-turn to pick us up from the other side, or just run us over for shakes and giggles. Some of us stopped in the middle lane to make sure that the express bus did stop before we cross over the last lane. We stepped aboard as if we were right on schedule.

Like most bus drivers running late, this driver put the pedal to the metal once we hit the freeway. When we quickly came upon the Page Mill Road exit, and still in the fast lane, people in the back of the bus started shouting directions. We cut through three lanes of traffic in a heartbeat. Once we were on Page Mill Road, the driver remembered the rest of his route. I arrived at my next bus stop to pick up the local bus with a few minutes to spare. Not surprisingly, a new driver learning the route with some coaching from the passengers.

 

The Python Time Zone Rabbit Hole

Thanks to the recent asshat controversy on Slashdot, and a fellow Slashdotter’s request for the link to the comment that prompted the controversy in the first place, I wrote a Python script to scrap my ~8,000 comments from Slashdot to dump into a spreadsheet for future reference. I’m planning to write essays about my various misadventures in Silicon Valley and my comment history is rich treasure trove of stories I’ve written over the years. While working on the script, I came across a programming rabbit hole for converting the timestamp string into a different timestamp string that kept me up for three nights.

The original timestamp that I extracted from each comment was a text string like this, “on Friday April 04, 2017 @06:03PM” (as it appeared on the website), and got written into CSV (Comma Separated Values) file just like that. After the initial script was working, I opened the 5MB CSV file in Excel to scroll through the data and see what I needed to change in the script. The timestamp string wasn’t in a sortable format. I had to change the timestamp into this format: “2017-04-07 18:03:00”. There’s two ways of doing this in Python: using a datetime object or slice-and-dice the string.

from datetime import datetime

def get_timestamp(string):
    return datetime.strptime(string, "on %A %B %d, %Y @%I:%M%p")

print(get_timestamp("on Friday April 04, 2017 @09:03PM"))

My first attempt (see code fragment above) was quite simple, using the strptime function for the datetime object to parse the timestamp string according to the matching format string (“on %A %B %d, %Y @%I:%M%p”). When I opened up the CSV file and compared the timestamp against the corresponding timestamp on Slashdot, the timestamp was correct except that the hour in 24-hour time was off by three hours. Every timestamps in the CSV file was off by three hours. I quickly learned that Python’s datetime objects are generally time zone unaware (or naive), and, in general, not very easy to use with different time zones.

def convert_timestamp(string):

    months = ['January', 'February', 'March', 'April', 'May', 'June', 'July',
              'August', 'September', 'October', 'November', 'December']
    month_number = {x: str(y).zfill(2) for y, x in enumerate(months, 1)}

    # remove "on" and split string into list
    string = string[3:].split(' ')

    # slice and dice into date/time components
    month = month_number[string[1]]                       # '04'
    day = string[2][:-1]                                  # '07'
    year = string[3]                                      # '2017'
    hour, minute = string[4][1:][:-2].split(':')          # '06' / '03'
    period = string[4][-2:]                               # 'PM'
    second = '00'                                         # add missing value

    # convert 12-hour time to 24-hour time
    if period == 'PM':
        if hour < '12':
            hour = str(int(hour) + 12).zfill(2)

    date_str = '-'.join([year, month, day])               # 2017-04-07
    time_str = ':'.join([hour, minute, second])           # 18:03:00
    return ' '.join([date_str, time_str])                 # 2017-04-07 18:03:00

print(convert_timestamp("on Friday April 04, 2017 @09:03PM"))

My second attempt (see code fragment above) was to slice-and-dice the timestamp string into the corresponding string values for month, day, year, hour and minute. The second value got added for completeness. If the period was “PM” instead of “AM”, the hour went from 12-hour time to 24-hour time. Date, “2017-04-07”, and time, “18:03:00”, are join together into one string, “2017-04-07 18:03:00” . When I ran the script and looked at the CSV file, the resulting timestamps was identical to the timestamps created by the datetime object.

Every timestamp was still off by three hours.

When I work on a website scraping script, I always save the scraped data into text files while refining the parsing and output sections to avoid re-scraping the website. That reduces the risk of my IP address being flagged by the website or firewall as a spammer and/or scrapper. The completed script will scrape, parse and write each page directly to the CSV file.

The slice-and-dice function converted the timestamp string as found in those text files. If I viewed the timestamps on the website, the timestamps are correct for the Pacific time zone. If I look at the timestamps in the text files, the timestamps were all off by three hours (“09:03PM” instead of “06:03PM”). So both the datetime object and slice-and-dice functions were working properly. The logical conclusion is the Slashdot server is located in the Eastern time zone and what I thought about the data was wrong. There lies the problem—and the solution.

from datetime import datetime
from pytz import timezone

def set_timezone(ts_str, tz_alt='US/Eastern'):
    ts_format, tz_def = "on %A %B %d, %Y @%I:%M%p", 'US/Eastern'
    tz_obj = timezone(tz_def).localize(datetime.strptime(ts_str, ts_format))
    return tz_obj if tz_alt == tz_def else tz_obj.astimezone(timezone(tz_alt))

timestamp = set_timezone("on Friday April 07, 2017 @09:03PM", 'US/Pacific')

print(timestamp.strftime("%Y-%m-%d %H:%M:%S %Z%z"))

The third and final attempt (see code fragment above) uses the pytz package to add time zone definitions to the datetime object and provide functionality to translate between different time zones. Using the datetime object from the first code fragment, the timestamps get encoded “US/Eastern” and then translated into “US/Pacific” to match the timestamp on the website. The resulting timestamp with time zone info in the CSV file has this format: “2017-04-07 18:03:00 PDT-0700”. A nice thing about the pytz package is that it also handles Daylight Saving Time seamlessly. If you don’t need the time zone info for the timestamp, remove “%Z%z” from the format string.

I Worked With A Murderer!

When I was a kid back in the early 1980’s, a teenager killed another teenager and hid the body in the foothills. The disappearance at that time, and the remains found several years later, was big news in the San Francisco Bay Area, one of several high-profile kidnappings and murders that prompted parents to keep their children indoors. Fast forward nearly 40 years later, a friend sent me the link to an old news article about the suspect arrested for that crime. The suspect in the perp walk photo was a former coworker whom I worked with for several years before he got fired, I thought, for being a douche bag. That wasn’t the whole story on his firing.

Unfortunately, I can’t share the article link to the article or the person’s name.

Although arrested and charged for the murder over a decade ago, the suspect was never tried as the case got thrown out of court for a lack of evidence—the murder weapon was never recovered—despite having a body and a confession. A person arrested but not convicted of a crime has a reasonable expectation to privacy despite the news media coverage. Or, maybe not. A recent court ruling for police taking DNA samples have implied that an arrestee  has “diminished expectations for privacy” as legal proceedings are public. Since I’m not familiar with intricacies of true crime writing (i.e., how to avoid being sued by someone who holds a grudge), I’ll be deliberately vague on the details to protect the innocent and the guilty.

The suspect and I started working together in IT support at the same time. Almost immediately he tried to put himself ahead of everyone else in a “Me! Me! Me!” attitude to impress the client—the company that hired the contracting agency to provide tech workers—in meetings, conference calls and emails. This insecurity, he told me, came from a deep need to avoid being unemployed again, which was understandable as everyone was still skittish years after the Great Recession. Except the “Me! Me! Me!” attitude never got dialed back as time went on and he never took the obvious hints from management to drop it.

Our first bump came when he accidentally rebooted the server that everyone logged into from our workstations. During the early days of the project, almost everyone rebooted the sever by accident while trying to reboot a remote system since the server wasn’t then properly configured to prevent admin users from rebooting it by accident. But he did it twice in one day. He never came forward to own his mistake. When the server owner checked the logs and fingered him in IM, he made excuses for why it happened—and those excuses went on for days. Everyone, including yours truly, roundly jeered him whenever the topic of accidental rebooting came up.

When I accidentally rebooted the server from a double-clicking mouse a month later, I contacted the server owner during the one-minute delay before the server rebooted, admitted my mea culpa to the team in IM (instant messenger), and replaced the double-clicking mouse. No one gave me grief for rebooting the server. The suspect complained loudly that he got treated unfairly because what I did was much worse than what he did. No one bought his story. That was the beginning of his reputation of being a douche bag.

When a team lead asked his group for volunteers to work with the suspect on an assignment a year-and-half later, no one wanted to work with him and everyone confessed that they didn’t like him. I felt compelled to write a long email to the project manager about all the problems I had with him. Two weeks later, he got fired. The funny thing was that no one told him that he got fired. According to coworkers on his team, he figured it out when his regular and admin Windows accounts  got deactivated, his badge stopped working, and the client refused to return his phone calls. Never did hear if security showed up to escort him out of the building.

The real reason he got fired, I recently learned, was an unknown coworker googled his name, came across the decade-old story about his arrest, and took the story to the client. The coworker got written up by the contracting agency for going to the client, but the coworker felt strongly enough that the client had to know who the suspect was. The client had a felony checkbox on the background check form (i.e., “Have you ever been arrested and/or convicted of a felony?”). The suspect, who didn’t live in a state like California with felony checkbox protections, failed to check the box and didn’t mention his arrest. That was enough to terminate his employment.

Have I Threatened To Shoot You Today?

I’ve read and commented on Slashdot since the dawn of the Internet (circa 1998). More so in recent years while waiting for a script to finish running at my tech job. I’ll find a topic that I’m interested in, read and respond to the early comments, and, if I want to torment the trolls, a.k.a, AC (Anonymous Cowards), I’ll write a controversial comment and camp out on the thread for the rest of the day. I don’t take this seriously because I’m just killing time. That is until an asshat accused me of threatening to shoot him. Even though I’ve asked three times for the asshat to explain how I threatened to shoot him, today I confronted the asshat by breaking out the crayons and coloring inside the lines.

What comment provoked this accusation? I asserted my First AND Second Amendment rights.

When talking about the U.S. Constitution, there are two groups that typically talk past each other all the time: the First Amendment people who don’t want the Second Amendment people bearing arms in public because they feel intimidated, and the Second Amendment people who loudly proclaim with obvious display of heavy weaponry that their amendment trumps all other amendments and that the First Amendment people should just shut up.

As a moderate conservative (another controversial statement), my belief is that you can’t have one without the other. The First Amendment grants me the right to speak my mind. The Second Amendment grants me the right to bear arms, and, since California isn’t a strong “stand your ground” state, I don’t have the right to shoot anyone’s sorry ass willy-nilly. This “best of both worlds” position typically pisses off the opposing camps.

One asshat ASSumed that my comment constituted a threat to shoot him.

If I was going to shoot that sorry ass asshat, I wouldn’t announce my intentions to do so under a named account on Slashdot. All the police would have to do is click on the home page link for my Slashdot comments, go to my author website and find my picture. The FBI already has my fingerprints. It wouldn’t take long to track me down.

So why draw attention to this controversy?

A group of Slashdot asshats went to my personal website, saw my picture and started calling me fat (among other explicit things). I collected their comments into an F.A.Q. (Frequently Asked Questions) and posted it on my website. Whenever someone called me fat, I posted the link to the F.A.Q in a reply comment and 3,000+ visitors stampeded to my website. That was 15 years ago and long before I had ads on my websites. This blog post is the new F.A.Q. If the asshat accuses me of threatening to shoot him again, I’ll post the link and collect the ad revenues from 3,000+ visitors.

After a two-year hiatus from blogging on Kicking The Bit Bucket, I’m blowing off the cobwebs and getting back to work.

Friday, 30 March 2017 — By popular request on Slashdot, I added a link to the original comment above. Here’s the link for the parent thread. Read and decide for yourself. That sorry ass asshat is still hounding me six weeks later.