iOS Configuration Profile & SCEP

I was recently tasked with creating an iOS configuration profile (.mobileconfig) that would submit a SCEP request to our Microsoft CA’s NDES server. The NDES server would then, in response to the request, issue a certificate to the requesting device. 

Easy enough, right? Well, as it turns out, no.

Everything worked fine up until the point when i tried to include the challenge pre-shared secret in the profile. If i included the challenge, the profile acted as if i hadn’t included it. The profile would prompt me to enter the challenge. I was able to manually enter the challenge and it submitted the request just fine. And i did receive the certificate back from the NDES server just fine. The issue was that the profile didn’t think the challenge was included in the profile. 

I used the Apple Configurator to create the configuration profile (.mobileconfig). (For the record, i also used Profile Manager and got the same result) Here is what the config profile looked like: 

Now, as you may know, a configuration profile is just an XML file. The Apple Configurator is just a nice GUI to create these XML files. Let’s take a look the code behind the above .mobileconfig. 

Take a look at line 8 and 9. That’s the Challenge. The reason iOS wasn’t seeing the challenge was because it wasn’t in the correct dictionary. For some reason, Apple Configurator was placing it outside the PayloadContent dictionary.

To fix the error, all i did was cut and paste lines 8 & 9 and move them down below line 36. Now the Challenge will be included in the SCEP request and all is well. 

The only other thing to note is that if you plan on signing your .mobileconfig with a code signing cert, you will need to make the above change before you sign the file. Otherwise you will FUBAR the file and the signature.

29 April Permalink

High Capacity Storage on a Budget

This will be a series of blog posts documenting a new project i’m taking on. It’s still in the planning stage, but as it progresses i plan to add more posts regarding the build and experience. Hopefully this is helpful to those of you out there who are looking to do the same thing. I had a lot of questions and wasn’t able to find nearly enough answers. 

Phase 1: Planning

Recently i was asked to find a solution for backing up my company’s Xsan. Seeing as how our Xsan is roughly 40TB large, my first thought was:

The storage arrays and xsan servers, by themselves, are fully redundant. Everything from the controller to the power supply has a hot-spare and/or backup. The storage is divided up into multiple hardware RAID6 arrays striped together into one large volume. This kind of redundancy, however, still leaves us vulnerable to destructive events such as fires, natural disasters, or floods. And that is why we need to backup everything. Preferably off-site or in a different area of the building. 

Goals

1. Cheap
   We spent the big bucks on the actual Xsan hardware. Performance and reliability aren’t big concerns here. We aren’t looking for a gazillion IOPS and don’t need 100% up-time. We just want something that will provide a good backup in the event the entire Xsan is physically damaged or destroyed. 
   
   
2. Lots of capacity
   We need something that will have enough capacity to backup the entire Xsan as well as provide enough capacity for future expansions. If all goes well, we are also considering using this solution for a few other purposes. (E.g. backup archival, VM snapshot archival, etc) 
   
   
3. Simple
   While we are willing to sacrifice reliability and performance for a cheap solution, we also don’t want to waste an inordinate amount of time fixing the damn thing. We understand that the nature of cheap/DIY solutions require some time and testing. However, when all is said and done, it should “Just work.” (© Steve Jobs)
    
4. Specs
   80TB - 150TB Raw capacity
   Gigabit ethernet
   Fast enough to keep up with a saturated a gigabit ethernet pipe 

Possible Solutions

Tape Backup

This was the first solution that came to mind. Currently we have an LTO4 tape backup solution in place for offsite backups of our critical data. For many reasons, including but not limited to the fact that tape backups are a giant pain in the ass, I’ve decided that tape isn’t the way to go here. While i’m sure tape is an awesome solution, it just isn’t my cup of tea. To each his own.  

Readily Available Storage Arrays

The second thing i did was take a quick stroll through the interwebs looking for solutions that are ready to ship. Even most of the “cheap” solutions were still a little too pricey for me. They also lacked some of the features and versatility we want out of this solution. I also wasn’t having much luck finding anything more than about 24TB that didn’t cost me my first born.  

Backblaze Storage Pod

This solution is pretty neat. If you’re not familiar with it, read up on it here. This was the first thing that came to mind when i first started looking for DIY storage solutions. Its cheap, all of the parts are spec’d and priced out, and Backblaze has done a most of the research up front. There are, however, several disadvantages to this solution. 

Disclaimer: This solution is purpose built and has a lot of very good use cases. For our purposes, these are some caveats that could cause us problems. I still think the Backblaze pod is a great solution and don’t want to diminish the hard work they put into their solution. 

1. Software
   The biggest reason that this solution works so well for Backblaze is their software layer that sits atop these storage pods. This solution is purpose built for Backblaze with their software in mind. Their software picks up where the hardware leaves off. Without their software, this solution leaves a lot to be desired. 
   
   
2. Completely non-redundant hardware
   While i’m not looking for 100% up-time, i am a bit concerned at the lack of some basic redundancy. As Joerg Moellenkamp points out in this blog post, the lack of power-distribution in the pod is a problem waiting to happen. As he points out, with the power-distribution they designed, there is no safe way to layout the disks. If you lose one PSU you’ll probably end up losing at least half of one of your arrays. Which, depending on your setup, could cause a lot of damage. 
   
   
3. Ease of access
   According to this blog post from Bioteam.net, the only way to get to the hard drives and other hardware is to remove 12 screws and open the top cover. In most racks this isn’t ideal or even possible. Not to mention the fact that they don’t include or suggest any sort of sliding rails. 
   
   This, obviously, means replacing a hard drive is going to be a giant pain in the ass. According to Backblaze, they replace about 10 drives a week in their fleet of 200-ish pods. That’s a lot of tiny screws to remove and insert every week. 
   
   
4. Cabling 
   One of the things I’ve consistently read about the Backblaze solution is that the cabling is a nightmare. There is no pre-made wiring harness available. This means you’ll need to buy the molex connectors and make your own. The other wiring nightmare lies in the backplanes. So many sata cables in such a small space. Unless you have mad cable folding skills, you’re going to have a bad time. 
   
   
5. Cost
   While Backblaze did list the cost of all the parts, they listed their cost when buying in bulk. Getting one or two of these items (some of them very specific) turns out to be quite difficult. And due to the demand on these specific items, they have increased in price. This means the actual cost of the pod is significantly higher than what they list. 

Overall this solution is cheap but lacks some basic redundancy features that i would expect even from the most basic DIY solution. Most of this is due to the design of the POD and the reliance on Backblaze’s proprietary software. I’m also a bit more hesitant to take on a complete DIY project that requires me to fabricate wires and other parts. These type of projects always take longer due to the fact that you have to constantly order tinny little parts and/or find temporary workarounds. 

OEM Storage Chassis 

When i really looked at the Backblaze pod i realized that the problem with their solution was the chassis and backplanes. The chassis needed to be easier to access, more redundant, and the backplanes needed to have better cable management. And it turns out that type of thing exists. 

• SuperMicro 847E16-R1400LPB
• Chenbro RM91250
• AIC RSC-5D

When you start looking at these chassis, you’ll see that you can get about the same drive density for roughly the same or less cost as the Backblaze pod. For example, if you add up the cost of the Backblaze parts (Using their price list. Realworld cost is going to be higher) that are included in the prefabricated SuperMicro 847E16-R1400LPB chassis, you’re looking at about $1922.  That SuperMicro chassis costs $1554. While the supermicro is only a 36 bay chassis compared to Backblaze’s 45, consider what you will gain.

The main con to this type of solution is going to be the lack of internet resources and experience. I’ve found several articles and blog posts talking about this type of solution but it’s nothing compared to the Backblaze solution. 

Conclusion

At this point i think i’m leaning towards to the OEM storage chassis solution. I’ll be sticking with a lot of the same equipment on the Backblaze list. But i’m having to alter the design a bit to compesate for the differing backplanes in the OEM chassis.  The main issue i’m still researching is the sata/sas HBA, expander and backplane interaction. Once i get that part figured out i think i’ll have a winner. 

Lastly, the main thing i’ve learned from this entire process is that each solution has a specific use case. You need to select your storage solution based on your specific needs. Each solution offers specific pros and cons and you must choose based on your situation. One size does not fit all.  

16 July Permalink

Fix for missing AD security groups in Lion

A few weeks ago i ran into a weird issue on some of our Mac OS X Lion (10.7) machines. For some reason they weren’t able to see all of our security groups under the Network Groups section. I’ve verified that this issue isn’t present in Leopard (10.5) or Snow Leopard (10.6) so i’m assuming this is a “new feature” of Lion. (As of 10.7.3 at least) 

Anyways, when you open “Get Info” for any file or folder and click the plus (+) sign in the Sharing & Permissions section you’ll see what i’m talking about. Select “Network Groups” and scroll through the list:

Missing a few groups, eh?

Here’s the problem. For some reason Lion only displays security groups from Active Directory that contain the “displayName” attribute. Newly created security groups, by default, do not contain this attribute. In order to get the security group to show up properly in Lion you need to fill in this attribute with the security group’s name. 

Here’s how you fix it. Open up the security group in your favorite Active Directory editor. (I prefer to use the one built into ADUC. You’ll need to check the “Advanced Features” option under the View menu to see it.) You will find that the “displayName” attribute is set to <not set>. Let’s fix that. Select the attribute and hit the Edit button and type in the name. 

Hit okay and then apply. BAM! Go to your nearest Lion machine and you’ll find that the security now shows up properly. 

As always, I contacted Apple regarding this bug “new feature” and have yet to hear back from them. I’ve combed through Lion looking for any hints as to why this is happening. For now you’ll just have to manually set this attribute for security groups you need on Lion until Apple releases a fix. If it really bugs you i bet you could write a powershell script to set the displayName attribute of all security groups in your domain. Just keep in mind that you’ll have to set this attribute by hand for any new security groups you create. 

Update: I just confirmed that new distribution groups created from within Exchange (2010 in my case) actually do have the “displayName” attribute corretly populated. So this may just be limited to security groups created from ADUC. (2003/2008/2008r2) 

18 February Permalink

Plex - My observations and experience

So yesterday i got fed up with the crappy Windows Media Center Netflix app and decided it was time for a change. My first thought was to check on Boxee’s Netflix app and see if they had improved it at all. Nope. It is still shitty as ever. Apparently they have a nice new clean version on the d-link box but aren’t planning to release it to the desktop software for quite a while. Someone on the forum mentioned they had switched to Plex for this very reason. Having never heard of or even used Plex i decided to check it out and see if it was any good. The following is my experience with Plex as well as some of my observations of the product. 

Before i begin, here is my HTPC setup:

My HTPC is a Core 2 Duo 6600 w/ 4gb of ram. It has a AMD/ATI Radeon HD 6450 graphix card. The box also has a Hauppauge WinTV-HVR-1600 capture card for capturing OTA HDTV. The box is running Windows 7 Ultimate 64-bit. I use the builtin Windows Media Center application as my primary HTPC interface. I use WMC mainly for its DVR functionality. From WMC i have several launchers installed that can open Boxee and Hulu. I use Boxee to watch all of my local media as well as media from sites like Crunchyroll and ESPN3. I use Hulu (Rarely) when i’m extremely bored and want to watch low quality commercials. ;) 

I should also mention that i’m spoiled when it comes to Netflix apps. I have a PS3 and used that Netflix app for a year before i setup my HTPC. I have yet to see any other Netflix app compare to the awesomeness of the PS3 Netflix app. 

With that out of the way, here is the rundown of my Plex experience. 

1) Plex doesn’t make it clear what applications you need to download and what these applications actually do.

When you go to their site and click on download (I clicked on the Windows icon) it takes you to a page with three buttons. It doesn’t really say what the three applications do. 

At first i downloaded the Media Server app only to realize it was just the server and not a player or client app like Boxee or XBMC. I immediately uninstalled the app and downloaded the Media Center app. Of course about 20 minutes later i realized that you need BOTH the server app AND the Media Center app for anything to work correctly. I actually stared at the following screen for at least 2 or 3 minutes before i realized i need to reinstall the server app. For some reason i was thinking this meant the app was connecting to a Plex server on the web. Not the case. 

To top things off, the Media Center app completely locked up and wouldn’t respond after I pressed Cancel. I had to end the task via task manager exit the app so i could install the server. 

2) The Channel Directory is completely broken.

Once i had the server app re-installed i launched the Media Center again and went to the Channel Manager to try and install the Netflix plugin/app. I clicked around to all of the different areas (Featured, What’s New, Most Popular, etc) and found the same blank screen everywhere:

The “Check for Updates” button even locked the app up and forced me to end the task and start over. 

3) Lack of documentation and helpful information 

Finding it hard to believe that this nice looking App could be so terrible, i began searching the wiki for answers. The wiki offered no help what so ever. I couldn’t even find a section for the Windows Desktop client. The only place in the Wiki/FAQ that even mentioned channels or plugins simply said “Access the ‘Channel Directory’ section of the client to find the channel you want and install it from there.” As i said, no help. 

I then decided to try the forums. After a bit of searching i finally found a post titled “0.9.5.0 - No Plugins/Channels.” After trying several different user’s suggestions i was finally able to fix the problem by deleting a random XML file from the server app’s Preferences directory. 

4) Inconsistent and broken experience

Upon opening the Media Center app i was pleasantly surprised to find that the Channel Directory was now properly listing apps. I quickly located the Netflix plugin and  installed it. The only problem was that after installing the plugin i couldn’t figure out how to access or even start the plugin. Nothing showed up on the main page. And when i went to the channel directory and selected the Netflix App all i saw this menu:

None of the listed options really did anything. I restarted both the Media Center app and the server app several times as well as the PC. Finally, after the PC restarted, i opened the Media Center and two new items showed up on the main screen. “Video Channels” and “Music Channels.” However, neither option listed the Netflix plugin. (Even when i selected both options to see more than just the recently used channels) They both just listed iTunes. 

At this point i was already 45 minutes into my very first Plex install and was very frustrated. I decided that this software wasn’t ready for public use and promptly uninstalled both the media center and server applications. And that’s where we are today. 

My observations as a geek

The idea of Plex is good. The problem is the implementation. The main problem i see is that the website lacks clear information on what the product is and how to use it. The front page has a bunch of unless marketing mumbojumobo and doesn’t even have a “What is Plex” type of page explaining what it does. 

Similarly, the website doesn’t tell you what applications you need to download or what the difference is between the various applications. Aside from offering a download button for the Media Center and Media Server, i could find nothing that described what they were used for and which i needed to install. And on top of that they didn’t bundle required software together as one would expect. (E.g. bundling the server with the Media Center) This lead to guessing games on my part and really irritated me. (Install, Uninstall, Install, Re-install, etc) 

The other thing that really irritated me were the basic bugs and interface problems as described above. This, to me, signals laziness on the part of the developers. These types of bugs are easily caught and squashed with a simple round of QA. I was able reproduce all of the above errors today on a clean test machine without even trying. (This is where my screenshots came from) It took me all of 10 minutes. 

Perhaps i’ll try Plex in the future when they finally leave beta. For now it doesn’t seem to be ready for the public. It feels more like a late alpha or early beta product meant for dedicated users who are willing to help the developers test and locate bugs. 

27 November 12 Permalink