Sony Flip 15 upgrade

I have a SONY Flip 15 convertible tablet (SVF15N17CXB) that I use primarily for contract work that requires a PC.  It is a very nice machine with an i7, 8G of RAM and switching discrete/integrated graphics adapter.  When I purchased it in early 2014 the two things it was lacking were an SSD and an active stylus.  I quickly remedied the lack of stylus.  It did come with a 1 TB hybrid fusion drive which I reasoned I would evaluate while SSDs continue to evolve.  I used this configuration for the better part of a year.  I noticed that most often observed slowness came back to I/O bound processes – the hard drive light was flickering a lot and the performance tab in task manager confirmed it.  So, with the last minute renewal of tax credits on business asset purchases I decided to upgrade the machine.

I had originally picked out a Samsung 840 EVO at 512G.  Since then the 850 line up was introduced.  Based on my observed hard drive usage I decided to go with the faster, but slimmer (in capacity) 850 PRO in 256G.

I figured if I am going to go through the work of opening the machine up, I would upgrade the RAM at the same time.  I went with 16GB of Crucial Ballistix that is also faster than the stock RAM.  Since, the hard drive I’m pulling out is a 1TB drive I picked up a USB 3.0 enclosure with UASP support so I can use it as an external drive.

20141230_195310141_iOS

Upgrading the Flip 15 is not for the faint of heart.  It is ultrabookish in form factor.  What that means is no easy access panels.  You get lots of screws with some of them hidden under rubber feet and a cover you have to pry off.  I found the tear down in this video very useful.  My tip is to start prying the cover off by the power connector.  Continue loosening the cover all the way around.  Slide the cover off by moving it away from the head phones connector.

20141231_021332939_iOS

In the picture above you can see the new memory in place in the lower middle of the picture.  The SSD is under the silver and gold colored ribbons in the lower right of the picture.  I did not utilize some of the existing foil/plastic shielding that was on the original case.  It appeared that it was primarily there as an insulator between exposed components on the HDD circuit board and the rest of the machine.  The SSD has no exposed parts.  It an all plastic case.  I did reuse the adhesive wrapper/shield on the bottom portion of the drive as it is also used to secure the drive cable connector.

Since installation of the new parts, the machines performance is much snappier.  Windows 8.1 Pro launches in ~4 seconds from a cold boot.  Development environments start up much faster.  Overall, much less time is spent waiting on the machine.

Today’s digital notebook experience

Let me start at the beginning.  I usually like to start in the middle or at the end, much to the chagrin of friends and family.

We have tried to replicate the notebook digitally—mostly successfully.  Let’s consider a few aspects of the digital notebook experience:

  • The physical manifestation of work.  The lack of the concrete.  The feeling of impermanence.  This can be overcome by making a hard copy.  But that just doesn’t make sense.
  • Instant access.  Today’s technology with instant on tablets and software like OneNote are addressing this.  And now with the cloud we can often access the same notebook from multiple platforms.
  • Search.  In a physical journal we can quickly and intuitively find content.  This seems to be a more difficult task with the digital notebook.  However, with a digital notebook you can use search against handwriting and simulate flipping through pages.
  • Poor vendor marketing of the active stylus.  In my experience, you must have an active digitizer to do good note taking or drawing.  With the exception of Microsoft Surface Pro and Samsung, even when a device has the capability the vendors are failing to identify the feature.  For instance the Sony split and HP Split X2s offers an active stylus option but you don’t really see that marketed well.  On top of that, many people don’t understand the difference between a $40 active stylus and a $5 capacitive stylus from the check out line.
  • Pressure sensitivity.  With the active stylus this has greatly improved.
  • Palm rejection.  This used to be a horrible issue on the dual technology capacitive touch and active stylus systems.  However, this has been mostly resolved.  I’ve used tablets from HP and Sony that this isn’t an issue.
  • The experience of the feeling of the pen or pencil moving across the paper is gone.  There is something magical about the feeling of a pen or pencil on paper.  Today’s active stylus typically has a very slick feel during use.  A long time ago, Cross made a stylus insert for their pens.  This was in the day of Palm Pilots.  It had a really nice feel to it.  It came very close to the feel of pen on paper.  I have seen no modern equivalent.

The digital notebook experience has come a long way.  With the wide spread adoption of tablet computing I really hope we continue to see advancement in the active digitizer field.  I would hate to see the technology lose to the far inferior capacitive touch only stylus.

Spike exploration: sftp, zip, config and Windows Service

I did a spike exploration to find out what is involved in creating a Windows Service with event logging. I also wanted to find out what is involved in working with sftp, zip and configuration with C#.  The code is available on github: https://github.com/k0emt/sftp-windows-service-spike

sftp

In order to test the sftp code I set up a Linux VM in the Azure cloud environment and created a user account on it.

I use the ssh component from http://sshnet.codeplex.com/ for sftp.  It is very straight forward to use this component.  The key for sftp is to use the SftpClient class.  Upload a file with the UploadFile method.  The sample code is in the ConsoleSftp projects Program.cs: https://github.com/k0emt/sftp-windows-service-spike/blob/clean/ConsoleSftp/Program.cs

zip

Use System.IO.Compression for creating/expanding zip files.

To use the ZipFile class, you must reference the System.IO.Compression.FileSystem assembly in your project.

http://msdn.microsoft.com/en-us/library/system.io.compression.zipfile(v=vs.110).aspx

I use ZipFile.CreateFromDirectory in the code above to zip an entire directory before sending it over sftp.

configuration

ConfigurationManager is the class that is used for working with application settings.

http://msdn.microsoft.com/en-us/library/system.configuration.configurationmanager.appsettings(v=vs.110).aspx

I used it in the service code to pull in a setting called user: https://github.com/k0emt/sftp-windows-service-spike/blob/clean/TransferService/TransferService.cs

In order for this to come together, you must create an app.config file in your project and change the property setting in Visual Studio to tell it to deploy when the project is built. https://github.com/k0emt/sftp-windows-service-spike/blob/clean/TransferService/App.config

Windows service with event logging

Create a Windows Service by choosing the Windows Service project template.

The Windows Service code must be installed and uninstalled.  It is possible to create an installer/uninstaller.  I opted to use the command line for this spike.

http://msdn.microsoft.com/en-us/library/y817hyb6(v=vs.110).aspx

http://msdn.microsoft.com/en-us/library/zt39148a(v=vs.110).aspx

I went here to learn about Service Methods – that is the kind of events to which the service can respond.  You can use the Visual Studio designer to enable the events you want handle.  Then you’ll need to code the events.  For this spike code I have it make an entry in the event log for each event type that is triggered.

http://msdn.microsoft.com/en-us/library/System.ServiceProcess.ServiceBase_methods(v=vs.110).aspx

Installing and Uninstalling the service

http://msdn.microsoft.com/en-us/library/sd8zc8ha(v=vs.110).aspx

Install with installutil.exe TransferService.exe

image

Go to Windows Services to see that the installed service is running.  See, the Spike Transfer Service below.

image

Uninstalling with installutil.exe /u TransferService.exe

image

Event Logging

For event logging a new EventLog is created.  An event source must be created and the source has to be wired up to the EventLog. 

Once this is done, the WriteEntry method can be used for writing to the event log.

You can go to the system event log panel to see where the events have been logged.

image

Summary

Overall, I was pleasantly surprised with the ease of accomplishing the spikes target tasks.  There was also decent documentation and examples available.

Oracle SQL Developer Date/Time format

One of the most super annoying things about Oracle SQL developer is that it defaults to a not so useful (at least to me) date format along the lines of 15-NOV-11.  I prefer to see something helpful like an  ISO 8601 formatted date/time.

The solution is to, in Oracle SQL Developer, go to: Tools > Preferences.

Then in the Preferences dialog, select Database and drill down to NLS.

In the NLS parameters you'll be able to set the Date Format to:
YYYY-MM-DD"T"HH24.MI.SSFFTZR"Z"

I have found that on some systems, you may need to use a setting like:  
YYYY-MM-DD HH24.MI.SS



References:

http://stackoverflow.com/questions/8134493/how-can-i-set-a-custom-date-time-format-in-oracle-sql-developer

http://orastory.wordpress.com/2007/05/15/dates-timestamps-and-iso-8601/

git: purging the lurking big file

I was just working with an old repository where I had inadvertently committed a large file ~300MB.  I subsequently removed it, but not in a way that took it out of history.  I discovered it when I went to push to GitHub and it complained.  These are the steps I used to get rid of the file:

  • git filter-branch --prune-empty -f --index-filter 'git rm -rf --cached --ignore-unmatch source/spikes/mo_offender_fak930.txt'
    -tag-name-filter cat -- --all
  • git update-ref -d refs/original/refs/heads/master
  • git reflog expire --expire=now –all
  • git gc --prune=now

Here are some pages I found useful to figure this all out:

http://stackoverflow.com/questions/6017987/is-there-a-way-in-git-to-list-all-deleted-files-in-the-repository

http://stackoverflow.com/questions/2100907/how-to-remove-delete-a-large-file-from-commit-history-in-git-repository

http://naleid.com/blog/2012/01/17/finding-and-purging-big-files-from-git-history

Engineer's Notebook

I record daily activities, notes, thoughts, diagrams, ideas, some reference material and special insights in their own paper notebooks.  If you were going to jot something down on a scratchpad, put it in here.  Document who you were collaborating with.  Why?  When I am in stand up meetings or talking with customers I go the paper route because I can simply open up the journal and write.  Also, it is less intimidating and error prone than having a computer between you and the people you’re talking with.

Use a pen.  Erasing and re-writing is too much work and will slow you down.  Simply, draw a line through the erroneous material and continue on.

When it comes to choosing a paper journal it is a very personal decision.  Many people like the Moleskine line of journals.  Ecosystem makes similar more colorful journals of the same format.  The things to consider when choosing a journal are:
  • Size.  Will I have it with me when I need it?
  • Do I want one that lays flat?
  • Does it utilize a closure of some kind?  Will it be a pain to open and close?
  • Does it have a pocket in the back for small papers?  Do I care about that?
  • Do I want plain, ruled or grid paper?
  • Do I want a few pages or lots of pages?
Personally, I use a journal that is made by Miquelrius.  It is about 6 inches wide by 8 inches tall and about an inch thick.  It has rounded corners and the pages are grid ruled.  Having the grid rule makes it easy to sketch out design ideas.  Another nice thing about this page size is that you can easily “scrapbook” in index cards or ever so slightly trimmed down print outs.
  
Create a margin on the outside of every page.  Put a date in the margin when you start a new entry.  You may want to add the time as well.  Some days there isn’t much to note and I’ll continue on the page with the next day.  In that case I draw a horizontal line between days to make it more visible.  The margin can be used for icon style call outs, indicating action/todo items, work partners, work times and post activity summaries or thoughts.

It is my experience that page flags get caught on things and torn up.  Bookmarks fall out and get lost.  Therefore, I recommend using Book Darts—small metal pieces that can be used to mark your page and point to a specific line.
With so many thoughts and notes recorded in your journal you’ll want to add your contact information inside the front cover with an offer of reward if returned.

I’ve experience quite a few benefits from keeping this journal.  For example:
  • Meeting action items and decisions documented and easily formally distributed.
  • Look at back at project notes from years ago and see what was done and why.
  • Able to reproduce work flow for prior infrequent work.
  • Able to identify collaborators for project work.
  • Identify when software was installed on a machine and what versions.
OneNote
For long term, shared electronic notes I utilize OneNote.  It is available on a wide variety of platforms now.  Notebooks can be local only on some platforms or synced to the cloud.  It lets you bring together your project notes into a portfolio style view.

OneNote provides numerous ways for getting information into it:
  • Typed and handwritten notes (it supports inking!)
  • Custom screen capture feature
  • Insert a picture from a file or camera!  (Use the camera to bring in notes from your journal.)
  • On Windows OneNote can OCR your inked writing, handwriting or print inside of images for search.
  • Print to OneNote
  • Drop in documents
  • Link to web pages
  • Voice notes
  • Drawings by hand and assisted
Compartmentalize
I would also suggest that you not use a single journal for all your needs.  Use one for work, one for the research you do outside of work.  In addition, consider a free form journal for capturing personal thoughts, events and accomplishments.

Summary
There are many good reasons to start and maintain an Engineer's Notebook.  Get a journal today and start the habit!

Strata 2014 Retrospective

Introduction

This year I was fortunate enough to be able to attend the Strata conference put on by O’Reilly and Associates.   I started this retrospective by doing a SWOT analysis of my experience.  I then expand with some logistics info and take away research ideas.
I had high expectations for this conference given its marketing.  The last conference of this caliber that I had attended was ÜberConf.

Strengths

  • The chairs in the session rooms were comfortable.
  • Healthy snack options were available at breaks.
  • Sponsored sessions were identified on the agenda.
  • There were lots of vendors at the expo.
  • Most if not all vendors had technical people on hand.
  • Mobile app had maps and schedule.
  • They did provide a daily print out of the session schedule and map.

Weaknesses

  • The mobile app was more frustrating than useful.  It was always wanting to update.
  • Being registered in the directory means you’ll start getting spam before the conference even starts.
  • Navigating the website to find specific session information was difficult.  Easier to find it through Google.
  • This is not an inexpensive conference to attend in terms of conference cost, travel and hotel expense.
  • Full day workshop
    • Late notice on software to pre-install
    • Not enough AC outlets
    • No tables!
Unless they address the logistics issues of the workshop environment I can not recommend attending one.

Opportunities

  • From key notes, sessions and the vendors you get to learn about what tools/processes the future holds.
  • Discern what tools/processes people are using now.
  • Talk with other attendees about the work they are doing and the approaches they are taking to it.
  • Some insights I had are:
    • Many are using Python and Julia for ETL.
    • R is being used for analysis.
    • Data people are starting to think about discuss data patterns.  Such as the Side Kick Pattern presented by Abe Gong (@AbeGong) a Data Scientist from Jawbone.

Threats (or why wouldn’t I want to attend)

  • Fundamentals are potentially better learned with targeted training.
  • Attending this conference could prevent you from attending another more relevant conference.
  • The target audience for this conference is narrow.  People that identify with big data, data science and business intelligence are well served by this conference.
When attending it helps to have specific questions or problems you are looking to solve.  This gives you a good context when choosing sessions and meeting with vendors.  (There are lots of vendors!)

Internet

The conference wireless was acceptable for as many people that were using it.  Internet in the hotel lobby was very good.  There is wired internet available in the hotel room for free.  I did not have an opportunity to use it.  There is also pay to use wireless available in the room.

Research

I came away from the conference with much I want to research and experiment with and people to connect.

Tools and Libraries

Techniques

  • Adjacency Matrix
  • pivot and fold operations
  • hexagonal binning
  • use visualization for data quality checks
  • confusion matrix
  • predictive modeling fundamentals
  • machine learning
  • The work of John Tukey (Statistics)

Speakers

Joe and Jeffrey presented: Data Transformation: Skills of the Agile Data Wrangler
Can we make big data management easier?  Her 3 research threads are: effective, easier and cost effective.

Summary

I came away with a better appreciation of what constitutes data science, the skills needed, the tools utilized and the vendors in the different areas.  If I attend again in the future I would likely skip the workshop day.  I would do additional prep working thinking about specific questions I may have for the technical people that the vendors make available.