Backups, Livedrive, FTP and Robocopy

So backups are important….right?  Really important.

If you don’t have backups, then you could easily lose your data. Hard drives will fail…it is not a question of “if”, but “when”.

For myself…I have been using Livedrive for a few years now.  Other than the occasional glitch (nothing major), it has worked well. One feature that doesn’t work well for me is the “Briefcase” – this is supposed to be a Local Drive letter that shows up on your local machine (usually drive L: ) and lets you access your Livedrive through normal Windows means.

I have several problems with it….1) I think I have too many files to let this work effectively and 2) the date/time stamps and 3) the use of *.ld files

1. This one is obvious – I have TB of data and they allow unlimited, so I am going to use it
2. I was using their FTP service to upload the files and as a result all the files are uploaded, but have the wrong date/time stamp in comparison to the originals
3. They only show the *.ld files and not the name of the real file

So…I figured I would use Robocopy (it comes standard with Windows 7 and Server 2008) to keep things in sync. Robocopy allows me to copy only the changed files over to the Livedrive, so it is fast and efficient …but like I said, the date/time stamps are wrong, so Robocopy wants to copy everything over!  Ugh. Plus, it sees the *.ld files as “Extra” on the destination side, so it wants to delete them.

I found a way to fix this and make it work the way it should.

I use their FTP service already, so I grabbed a copy of Webdrive. This program turns a FTP site into a drive letter, but it uses standard FTP commands, so no funny business. Secondly, if you check the reference for Robocopy, you can see some advanced features – specifically, /TimFix, /XO and /XN.

So…to get things in sync, assuming a mapped drive of S: for my server and a W: for my Webdrive, I ran:

robocopy S:\Photos W:\Photos /E /TimFix /XO /XN

This updates all the Timestamps on the destination without actually copying anything over.  /TimFix says: FIX Timestamp on all existing destination files, including skipped files.

Perfect! Now the two systems are in sync and I can use standard Robocopy commands to upload new/changed files to Livedrive:

robocopy S:\Photos W:\Photos /E

This will copy only changed or newer files (including subdirectories) over to my Livedrive.

Thus I have a copy locally and a off-site backup in case something happens.

Jim

Exchange, Outlook 2007, Outlook Anywhere and Self Signed Certificates

To all those people that say you Can’t use a self-signed cert and be able to connect to Exchange using Outlook Anywhere, I say pfffff***.

I just did it. See near the bottom for instructions – basically you need to import the self-signed cert into your Trusted Root Certification Authorities.

In case you don’t know….Outlook Anywhere allows you to connect your Outlook Client to Exchange over the internet without having to VPN in or anything. It is still secure because it tunnels everything over the HTTPS connection that you would use if you were using OWA (Outlook Web Access). This means you can get internet somewhere and just start up your Outlook and get connected and have full functionality! It’s great!

If you don’t want to buy a SSL Cert, you can use the Self-Signed one that your Exchange installed – it is just as secure – and you know it is safe (even if your computer doesn’t) because you are the one that signed/created it!

Anyway….after you get Exchange setup with Outlook Anywhere (on 2010, you just have click a check box!), you need to setup your Outlook client. There are lots of examples on the web on how to do this part, but here are some quick ones:

On the Exchange Settings screen, put in your INTERNAL EXCHANGE name (not the externally known one), and put in your user name.

Click More Settings.

In the Exchange Settings dialog box, select the Configure Outlook Anywhere check box and then select the Connect to Exchange Mailbox using HTTP check box.

In the text box that follows these check boxes, type the server name for the Outlook Anywhere proxy server. Do not enter http:// or https:// as part of the name. This will be the external name – should be the same as your OWA like this:  mail.yourdomain.com

Select Mutually authenticate the session when the system connects with SSL and enter the principal name of the proxy server. This will be like this: msstd:mail.yourdomain.com

Select whether or not to reverse the default way in which Outlook decides which connection type to try first, LAN (TCP/IP) or Outlook Anywhere (HTTP). The default is LAN (TCP/IP) first, then Outlook Anywhere (HTTP). If you expect users to connect when they are outside the corporate network more frequently than when they are inside the corporate network, we recommend that you configure Outlook to try Outlook Anywhere (HTTP) first.

Select an authentication method from the drop-down list. If in doubt, use Basic Authentication.

Click OK to return to the Exchange Settings dialog box, and then click Finish.

Now here is the tricky part – it all won’t work until you import your self-signed cert. To do this, start up IE and browse to your OWA where you will get the Cert error. Continue anyway.

Then at the top of the browser window, you should see “Certificate Error”. Click this and then click, “View Certificates”. Next, click Install Certificate. Now choose “Place all certificates in the following store” and click Browse. Choose Trusted Root Certification Authorities and click Ok. Once you click Next and Finish, it should say Import Successful. Now if you close your browser, re-open it and go back to your OWA URL, you should no longer get the Cert warning.

AND…best of all… your Outlook will now connect to your Exchange without a VPN connection!

Enjoy!

Jim

VMware View 4.5

Two things: They took out the ActiveX control out of the View Client (hopefully they bring it back later on), and the View Security Server ONLY SUPPORTS RDP – NOT PCOIP.

I setup a whole environment for a client and was testing away, only to find out that the Security Server (the part that goes in the DMZ and handles the Web redirects to the View Server) does not do PCOIP. The VMware docs do mention it, but only a single line that says that the Security Server will be ignored if PCOIP is picked as a protocol (which is the default too) on the View Client.  This is probably not the greatest arrangement and I am sure VMware is going to need to make adjustments – if only to allow the View manager to set a different default for the Security server than for the regular View server.

Everything else with View 4.5 seems to work great.

Jim

Windows 7 – Took the plunge

I have been resisting for a long time, but my work gave me a brand new laptop with Windows 7 Professional preloaded on it.

So…rather than downgrade I thought I would finally get used to it and really take the time to give it a fair shake.

Yes…they moved things and getting to some items is not as easy as it used to be, but they improved some things too.

Like this Windows Live Writer – maybe this was around and available for XP too, but I didn’t know about it. I actually only found it by accident:

I am away from home on work and my wife runs a little book publishing business (shameless plug – www.yournickelsworth.com). Anyway…she wanted me to update our website with a review of a book that she recently finished. She sent me a .DOC file and a massive 2MB .jpg file.

I just got this new laptop (as I said) and I don’t have Word nor Photoshop – which I would normally use to massage into .HTML and then post onto the website. But, didn’t have any of those tools. So…..

I knew that if I emailed the .DOC file to myself at my gmail account, I could view it as HTML, so I thought that that would be a good start. But it turned out to be easier than that. After receiving the email at gmail, I was able to edit online! Cool stuff. Then there is an option to “Share” with everyone…so I did that.

Next, ...I figured out a way to resize the 2MB jpg even without Photoshop....I started editing it with the builtin Windows 7 tools, and I noticed an option to make it into a Blog post (this is where I found the Windows Live Writer thing). So, I posted it to my blog which automatically resized it and then I download that image and then added it to the Google Document I had been editing.

And Voila…I now had a page that I could link to and share with everyone on the web. I added the link to my wife’s website and I was done.

Now that I know how to post to my blog using the Windows Live Writer and how much easier it makes it, I will probably blog more often.

Jim

Norco build update

So I finished the first OCE/ORM (Online Capacity Expansion/Online Raid Migration) using the Highpoint 4320 RAID card. I moved from a 3 drive RAID 5 to a 4 drive RAID 5 as part of my move from the old WHS server to my new Server 2008 build.

Other than it took 2 days, there were no problems. At the end, I had to go into storage manager on the box and "Extend" the size of the disk. 2 important points that allowed this to just work:

1) I had already made it a GPT disk (so it could handle very large partition sizes)
2) I had already made it a Dynamic disk (so it could be easily extended)

The other thing I did was to install Windows Media Connect 2.0 on this server so that I could share my files out with my XBox 360. My Windows Media Center didn't need this as it can use the file shares I already setup.

Here is how I did it:

1) Find a copy of Windows Media Connect 2.0, this is no longer provided my MS but can still be found around the web. Another place you could potentially find it is in the file umupnp.exe on the Windows Home Server installation media.

2) Now out of the box this won't install on Server 2008, so don't bother trying, you need to extract the files from the archive. I used Winrar, other archiving tools will do it too.

3) Create a folder, for example C:\Program Files\Media Connect (I'll assume you used this folder for future examples)

4) Copy the following 3 files which were extracted from the archive mentioned in the last step into your new folder: wmccds.exe, wmccfg.exe, wmcsci.dll

5) Run regsvr32 "C:\ProgramFiles%\Media Connect\wmcsci.dll"

6) Run "C:\ProgramFiles%\Media Connect\wmccds.exe" -installwithfiles (This is an undocumented feature of Media Connect which installs the service without paying attention to pesky details like OS version).

7) Run sc config "WMConnectCDS" depend= none
NOTE the space between = and none...it is required!

8) Run sc config "WMConnectCDS" depend= upnphost/HTTP (These 2 commands are essentially an easy way to remove the HTTPFilter service dependency which does not exist on Server 2008. It's functionality now resides in HTTP, I could of done this from the registry as well but in my experience that requires a reboot). NOTE the space again!

9) Enable the SSDP Discovery Service by setting it to manual startup from the GUI or using
sc config "SSDPSRV" start= demand
Don't forget the space between = and demand!

10) Enable the UPNP Host Device Service by setting it to manual startup from the GUI or using
sc config "upnphost" start= demand
Do I really need to say it again?

11) Run wmccfg.exe and do the initial setup.

Now you have a icon in the system tray you can click to open the configuration to make changes, or you can run the wmccfg.exe to reconfigure.
 
Next I am going to move the rest of my files from my WHS and then perform another OCE/ORM to RAID 6 with the remaining drives.
 
Jim

WHS (sort of) and Norco Build

So I rebuilt my WHS (Windows Home Server) - sort of.

Originally (see my previous post), I was going to make a new server using the Norco 4020 (I know the post says 4220) and a Highpoint 3540. The idea was to get the lowest cost per port. The 4020 case uses SATA cabling on the backplane and the 3540 RAID card comes with a whole bunch of SATA cables, so I would have everything I need to hook it all up.

The issue is that the 3540 handles 16 drives, so I would have to do something else to handle the last 4 drives. Plus, the 3540 is almost $700 CDN.

I didn't know much about SAS Expanders, but I had heard about them. So....after more investigation, I decided to get a Highpoint 4320 raid card instead. It was $482 from Newegg.ca and has 2 SFF-8087 Mni-SAS connectors. These connectors breakout to 4 Sata connectors each. But then you say, "I thought you wanted 20 drives?!".

True - but the 4320 can handle SAS Expanders - the 3540 can't. With SAS Expanders, the 4320 can handle up to 128 drives. I can take one of those SFF-8087 connectors and turn it into 16 (or more) drives. I just need a SAS Expander card. The Chenbro CK12803 is a compatible SAS Expander for the 4320. The Highpoint 4320 also comes with 2 SFF-8087 Mini-SAS cables.

So...I found the Chenbro at http://www.ncix.com/ for about $280 CDN. That means the total cost of the RAID card combo is now $762 - about the same as the 3540 was going to be, but now I have more expandability, plus I am now going to use the Norco 4220 case which uses the SFF-8087 connectors on the backplane, so I will have much less cabling.

I was gonig to buy everything from newegg.ca, but NCIX often has sales on hard drives and they price match, so when I called them, they were willing to meet the pricing I got from Newegg on the case etc. The only thing they didn't have was the Highpoint RAID 4320 card. So here is my shopping list (prices all in Canadian):

Norco 4220 Case - NCIX - $339
3 extra SFF-8087 Mini-SAS cables (Norco brand) - NCIX - $14 each - $42 total
3 WD 2 TB hard drives - NCIX - $99 each - $297 total
Chenbrok CK12803 - NCIX - $280
Highpoint 4320 Card - Newegg - $482 - comes with 2 SFF-8087 cables
ASUS KPL5PM Socket 775 motherboard - local computer store - $60
2 GB RAM - local computer store - $40
Intel E5400 CPU - local computer store - $65
Coolmaster 750W PS - local computer store - $82

Total: $1687

I already had 3 2 TB WD hard drives, so now I have 6 total - plus I have a few 1 TB and 500 GB drives lying around.

I went cheap on the motherboard, CPU and RAM as I didn't feel I needed to worry about speed too much. Nor did I really need alot of PCI slots. And the board had built in video and a gigabit lan connection. The thing I DIDN'T want to cheap out on is the Power Supply. This is going to be my main storage unit with my only copies of files (although I do use http://www.livedrive.com/ for off-site backup). I could have bought a $14 power supply, but then it would have been the weakest link and I didn't want it to cause any issues.

After I received the gear I began putting it together.

I have built several NAS boxes before using Chenbro cases and Areca RAID cards - they are nice, but the total cost would have doubled.

I was fairly pleased with the build quality of the Norco case - considering the price. The only thing that seems a little "cheap" are the drive trays. They work ok and fit nice....I just don't think you want to be rough with them - they won't take it. The Chenbro drive trays are much more robust in comparison.

When I put the motherboard in, it looked very small in the space - you could easily fit a much bigger motherboard with many more slots.

The Highpoint 4320 needs a PCI-E x8 slot and is compatible with the x16 slot usually reserved for the video card, so I used that. The Chenbro CK12803 fits in any slot. It doesn't really use the PCI bus at all - the slot is just to hold it in place. It gets power from a standard power plug.

The space between the fans and the hard drive backplane is fairly tight, so getting the power plugs in was a little bit of a challenge.

Everything else fit nicely and I had no issues plugging everything else in.

I had read a number of comments that the fans in this case are really loud - so I was prepared. As I indicated above, I have built a number of NAS boxes before and they can be quite loud.

However, when I powered on the machine, I was pleasantly surprised. It didn't seem that loud to me. I checked it with a DB meter and at 1 meter (3 feet), the rating was 58 db. I am putting all my stuff in my heated garage anyway, so I didn't really care - I can't hear any of it.

Next I installed 2 500 GB drives in a mirror to use for boot drives. I decided to go with a standard Windows 2008 install instead of a WHS (which is why the "sort of" in the title).

The other reason to go to Windows Server 2008 is the new SMB version 2.1 - it increases speed of transfers when going from/to a Windows 7 machine.

I am keeping the old WHS around in a much smaller capacity in order to handle backups etc.

Then I created a RAID 5 from the 3 2 TB hard drives. I set the RAID 1 Mirror as boot in both the RAID card and in the Bios, booted the machine and loaded Windows 2008. I had to have the drivers on a USB in order to load them during the installation process, but other than that, it was all smooth.

Once I had the system running and performed the standard windows updates, I enabled the guest account so that I would have no issues with sharing the folders with various machines - and Xbox 360 - around the house. Yes, that is a major security issue, but this machine is safely isolated behind my firewall and no one outside of my system has any access to it.

Next I performed some performance testing.

WOW! Was I impressed. On the Raid 5 and with only 3 drives, I was not expecting massive performance, but I got a consistent 250 MB/sec transfer rate. Yes...that is 250 megaBYTES per second. Since I only have a single gigabit lan connection, I will be liimited to a max of 100-120 MB/sec, but that is still plenty fast. I may get a dual NIC Intel card and bond the two channels together to get a 2gb lan connection later on.

Here is the output from ATTO Benchmark:

I am currently moving all my data from my old WHS to the new server and all is going well. As I move the data, I am removing the old drives from the WHS and adding them to the 2008 server. I am using ORM (Online Raid Migration) and OCE (Online Capacity Expansion) to allow me to move to a different RAID (RAID 5 to RAID 6) and to allow me to expand my disk size without taking anything down. This will take several days to complete, but I have the time.

The last thing I am doing with this is to install the free version of the Starwind Software iSCSI target. This allows me to use this box as a storage server for my small VMware ESX (vSphere) farm where I keep all my virtual machines running (SBS, Asterix PBX, Mailcleaner Spam and a Linux server that is a LAMP box).

I am going to end up with 6 2TB drives running a RAID 6 (total of 8 TB of space) and I will have 12 empty drive bays that I can use for expansion.
Overall I am extremely pleased with the result and would highly recommend this solution to anyone else looking to build an inexpensive storage server in their home or small business.

Jim

XVid - the best overall format

I have recently setup a Windows Media Center....and I have a Samsung Blu-Ray player that has a ethernet port and supports DLNA....and I have a XBox 360 - which can act as a Media Center Extender.

The problem is that I was having trouble getting movie files to play in the best quality and with 5.1 surround sound on all units.

If I use WMV-HD, then it works fine on the XBox 360, but the Media Center and the Samsung give me only stereo.

If I use MKV, then it works fine on the Media Center and Samsung, but the XBox 360 won't play at all.

I finally found that using XVid with AC3 sound is the format that is compatible all the way around. It was actually a complete fluke that I found this.

I happened to have a hi-res copy of Terminator 2 in a .AVI file and it played fine on all 3 units, so I used GSPOT to find out exactly what format it was.

What I found was this:

As you can see in the highlighted areas - the Stream Type is using OpenDML (very important), the codec is xvid, the resolution is high-def (basically 1080p or close to it), the audio is AC3 (5.1 surround sound) with 6 channels at 48Khz and recorded at 640kbit/sec (good audio), and it looks like Virtual Dub was used to create this.

So....that gave me the info I needed to make this happen.

So...to convert a WMV or MKV, you first use VirtualDubMod (you have to create a .AVS text file so you can open either one - this file just has the path and name of the file like this: DirectShowSource("C:\yourfile.wmv") ) and open the .AVS file. Then go to Streams and click the Save WAV button. You will have to wait a while (maybe up to an hour) and you will need lots of disk space as it is creating a RAW WAV that contains the 5.1 audio.

Next, you use eac3to (Google it) to convert the WAV to AC3 audio - this part is pretty quick.

eac3to input.wav output.ac3

Next, you go back to VirtualDubMod and in the Streams section you Add the .AC3 file you just created. Also...Disable the original audio that was contained in the file.

Next...use the File->Save option and change the compression to Xvid codec - I set my quality to 3 - seems to make a decently sized file while retaining quality.

Then let it run for 3-4 hours.

Voila! You now have a HD Xvid that will play on just about everything with 5.1 audio.

I don't know why more movies aren't being posted as 1080p XVids...this format seems to be great! With a quality comparable to MKVs. You can't do DTS - and subtitles are a bit more problematic - so maybe that is why. I see a lot of 720p XVids, but not 1080p.