For those of you that don’t know, Amazon’s S3 is a paid service for offsite storage that charges really competetive rates for data storage and transfer. I recently wrote a post about MySQL backups in general, and mysqldump specifically. Someone just took the solution I wrote about to a whole new level.

This link to Shanti Bradford’s blog YARB explains how the software works and gives examples for the very simple configuration. I am in love with this solution for smaller databases or funded projects with a few dollars to spare for a super clean offsite backup solution. Leave it to a Rails person to find the easiest and best solution to a problem that everyone has! Big ups Shanti!!

Bookmark and Share

 

How do you spend your time at work? If you’re a DBA, you should be spending it well. There are so many responsibilities that you will face as a DBA. Sometimes, you’ll need to put the time into tedious and difficult tasks while remaining interested and engaged, other times you’ll need to keep a certain mindset or attitude to maximize cooperation from coworkers. The information below is not a complete roadmap to success, but it is a great set of guidelines for someone looking to become a DBA in a small or midsized organization.

 

To begin with, and perhaps most importantly, you can spend much less time maintaining and tuning your systems if you understand the basics of relational theory and design principles. In order to gain the benefit of decades of research in the relational field, make sure that you are well aware of best practices, good relational design, and the basic fundamentals of databases in general. This is the biggest mistake amongst DBAs. Know your craft at a fundamental level, or be forever forced to repeat the mistakes that we see every day in the MS SQL programming newsgroups.

Once you are sure that you system is well designed, you need to check on your systems, relentlessly and with no mercy. I spend at least an hour per day either checking on systems, or automating system maintenance and observation. This includes creating new alerts with various triggers, automating DMV reports to myself, and running maintenance via direct SQL or the SQL Agent. A lot of the time, a simple manual check on a machine can unearth problems that you haven’t imagined. When you find these, modify your maintenance plan and ensure that the problem doesn’t find itself happening on other SQL instances. It is important that you visit your machines and have a look around occasionally.

Performance tuning your system is closely tied with the monitoring of your system. As you find problems and bottlenecks, solve them. Take care of your indexes and your code and take care of your performance in general. Ensuring that you can scale in the event of unexpected and unpredictable growth will allow your employers to make the kinds of leaps in growth that will keep them healthy. A healthy employer experiencing growth is much more likely to throw a poor DBA a bone when bonus time rolls around.

For further performance tuning I spend time every day rewriting queries for performance. Going over my old code allows me to grow, and utilize new methods that I learned through my research. Rewriting code is probably one of the best things you can do to improve your programming skills. Make sure that you know how to measure performance, so you can verify that your changes are actually improvements! Additionally, there are other people here at my office writing SQL. Developers, report writers, and management are often rolling their own pet projects that often are eventually moved into production. I have to make sure that their code is either portable, or at least documented. Some of this code is really bad, and can take an inordinate amount of time to fix it depending on the talent of the original author.

You must research new technology and read up on blogs and SQL newsgroups every day. Keeping up with the newest published material is also a necessity. It is very important to keep up with not only the newest developments in the industry, but also to go through older material to ensure that you haven’t missed anything important. A good DBA will be a voracious reader. Just last month I implemented some calendar tables after reading Celko’s newest book (featured on the sidebar to your right.) I try to read most of the new titles from major authors in the field like Celko, Delaney, Ben-Gan, and Date.

All that knowledge is worthless without execution. For a quick solution in an emergency, testing is very important! I spend a few days per month testing restores, disaster recovery, and basic administrative functions. Databases are mission critical in almost every case, and you need to make sure that your skills are sharp in case of a problem. Regularly practicing the scenarios listed above will put you miles above the rest when it really counts. I extend this to reviewing the names and functions of valuable troubleshooting DMVs, reviewing the location and purpose of my collection of SQL, and reviewing my ability to fix locking and performance issues. Do you know how to connect into the DAC and kill an out of control process if your CPU is pegged? Could you do it without looking up the processes and methods? If your DB is having major issues and people want it up right now, will you be distracted with your nose in a book, or fixing the problem?

Paying attention to specifics is required in any job, but we lose our way without the big picture. As a database admin, or any IT professional, evaluating and implementing sweeping trends in IT are important if you want to stay on the cutting edge. Green IT and server virtualization are quite big right now (and somewhat related to each other.) I can proudly say that my organization is on the bleeding edge here. You should examine these trends with great care to ensure that you are not missing out on the next jump in technology.

New technology inevitably leads to new projects. New projects are a major draw on my energy, and any rollout of a new system, especially when written in house, takes all my energy in design, planning, and implementation. If you’re like me, most of your time will be spent on supporting new databases and new code! Make sure that you spend some time each month planning for future growth and really putting hard numbers on paper. If you think you need to move to a new SAN in 6 months, tell your CFO and CEO now. They’ll work it into their projections, and you won’t be left caring for an inadequate system at the last moment. Not planning for the future always looks bad and inevitably ends up costing productivity for the entire organization, so even if you’re busy you need to do this.

In the absence of a major project being implemented, or perhaps due to requirements of an upcoming project, we all have to upgrade. You’ve got to keep up! Patching and upgrading don’t happen often, but when they do, they’re a huge drain on my time. Planning a rollout to 22 SQL servers littered all over the globe can be an enormous project. Planning and testing are key here, and should never be rushed. Although I’ve been working with SQL 2008 for months, I estimate that my half completed plan will take me another few months to design, and probably a week to implement. When I do implement my rollout, I’ll be forced to update nearly all of my documentation.

Documentation is a difficult process. Not only does it require you to spend what could very well be years on the inevitably poorly documented existing infrastructure, but any time a change is made the appropriate changes to documentation must be made. Using a change management system is very helpful in this respect, but will often inspire angry e-mails and hateful glances. Ignore these and do what you need to do to ensure that your system is properly documented. Brush up on your UML, or Visio if UML is too much, but never avoid documentation.

 

Complying to international standards like ISO, and local standards like a naming convention for objects, sounds very easy. It never is. Unless you are the conduit through which all changes in your databases are made, you will need to clean up other people’s messes. Stay steadfast in this, for it will make all the difference should anyone ever need to work on your system without you there. It will also ensure that your system is as portable as possible when the inevitable change happens. It will also make documentation much simpler, due to the standardized nature of your work.

One thing that most DBAs let slide until well after a production system is rolled out is : security. Security is so important these days that I spend a few hours at least once or twice per week checking existing security, as well as really taking the time to think about how the system should be used, and allowing my security to reflect that. The days of global dbo are over, and you’re inviting trouble if you let your security wane. Not only are you protecting yourself against malicious destruction, but you also need to protect against mistakes and well meaning but fatally flawed design changes. You should have a security plan for every database in your organization, and that plan should allow people to do ONLY what they need to do. Less is more, always, in access rights.

As for working relationships, be nice to your developers. Spend some time programming so that you can understand their side of the story. A good working relationship with your developers will ensure that they are willing to compromise when it counts. Sometimes, you need to compromise too, just make sure that you only compromise on things that don’t affect the integrity of your system. I find that delivering bad news to my developers is always easier after we’ve spent a recent Friday night at a concert or a bar.

To further your relationship with your management team, consider introducing them to Business Intelligence. Any organization with a large amount of data will want to analyze that information. Make sure that you are proficient with ETL processes, and report writing. If you have a BI guy or girl, they will likely take care of this for you. If you don’t have one, spend some time on BI and show your boss what a database can do in the right hands. Don’t worry about overloading yourself, be firm that if they want to continue on the BI path, they’ll need to hire someone else. They’ll evaluate the situation and decide if the additional cost is worth it. It almost always is, and you’ll be the new IT hero.

I think that if you follow the above to the letter, you will definitely be the best DBA in the world. Unfortunately no one person can hope to achieve it all at once. Hopefully you have some help, but if you don’t, don’t lose hope. Do your best and do as much as you can. Take pride in your work, as that is without a doubt the single most important thing that you can do.

 

 

 

Bookmark and Share

We had a few people over last night to check out Kalen Delaney’s new  DVD from http://sqlserverdvd.com. I decided to have a database movie night and invite a few professionals from the Las Vegas area where I live.  I invited about 10 people, but due to my poor planning and short notice only a few were able to attend.  The group was distributed pretty evenly among the many faces of the DBA: a BI specialist, a software developer with responsibilities in SQL Server, and a guy who does tech support for about 150+ sites running SQL 2000/2005 but doesn’t really ever have to deal with anything more than adding accounts or moving data around.  At around 7:30PM on a balmy spring evening we plopped down on my oversize couches, snaked power cords all over the living room, calmed my dog down and got to business.

The DVD, for those of you that don’t know, is the first in what appears to be a 10 part series from Kalen Delaney that essentially mirrors her 5 day course.  Imagine, getting a 5 day training course from a seasoned professional for about $300.  That is exactly what you’re getting, without the crappy hotel and bad coffee.  Chuck Boyce has built a little community around the project and in under a few months has already recruited a new player with a new DVD.  Being that the best video training for SQL Server has been amateur and uninspired up until now, we’re seeing something really important happen.  The majority of the people who don’t like reading have been forced to in the past to learn by experience or through a mentor.  For what may be the first time in history, people who don’t like to read and don’t have the money for expensive personal training can become the kind of DBAs that the world needs : capable, well informed, and extremely knowledgeable about the details that keep the software running.  After looking at the course outline as a group, there are some subjects that people were positively frothing for…especially indexes!

The DVD was great in that there was something for everyone. Despite having read Kalen’s book about the SQL storage engine, I myself found some new and interesting information, and I think everyone in the group walked away with some information that was important to them. The video also did something somewhat unexpected, for me, and that was it raised questions about not only the content of the video (which were easily explained by at least one person in the group) but also regarding real life scenarios that were only loosely tied to the topics of the video.  People were discussing everything from strange lockout and error issues on testing servers with only two users (hint : temp tables are BAD) to asking questions like “Just what the hell is socks and how does it apply to SQL?”  (Once we figured out he was talking about Sarbanes Oxley, the explanations came quickly and easily.)

I found that about half of our four hour viewing time was spent paused and talking amongst the group.  This is a really great way to watch the DVD as it keeps the topics lively and helps you to see these lessons from perspectives other than your own.  Kalen gives you the facts, but it is up to you to creatively use the information to your benefit.  While she often gives examples for useful ways to apply her knowledge, there is no way that she can cover every possible situation, but you can get a lot closer with 4 people from different jobs putting their brainpower into figuring out how to best utilize the information.  This is not to say that you won’t get anything out of watching it alone, you’ll just have to think a little harder and watch it a few more times.  I plan on watching the DVD one more time just to make sure that I didn’t miss anything.

We spent a lot of time paused and discussing the ramifications of the DVD, especially what to do with the DMVs and how they could apply in the real world for someone like a developer (Pro tip : it is a great way to get your boss to upgrade your development machines!!)

Kalen’s speaking voice was great and she didn’t stumble much, like most presenters do. She is obviously confident and comfortable in her knowledge and being in the public eye.  She doesn’t have any strange speech patterns to distract you, and she has no facial or vocal tics which makes it very easy to watch her and focus on the material at hand.  Nothing drives me crazier than a presenter or teacher that takes over the stage and puts the subject matter in the background.  Kalen understands this, obviously, and while being fresh, personal, and well spoken she is not the focus of the lesson, the subject matter is.  One thing I did notice was that the sound changed dramatically from the lessons to the demos. I’m not sure if it was going from mono to stereo or from stereo to surround, but it was apparent on my surround setup.  Audiophiles beware, but for everyone else I recommend laughing it off and pointing fingers, because you probably won’t hear it over normal TV or computer speakers.  As for the production values on the rest of the disk, they’re great.  The picture was clear on my TV, there was no hum or static, and Kalen’s voice was right up in the front rather than her keyboard, her CPU fan, or anything else that you commonly get on instructional DVDs.  You won’t be disappointed with the quality, and it really does give you a feeling of being in a classroom environment.

It is great to see some decent video training for once.  This is a great thing for the community.  Most of all I’m excited to see the people that I know with a passing interest get more involved in the future of the database-fu, and I can only imagine that this kind of enthusiasm will spread if we get enough enthusiastic database pros passing the url to SQLServerDVD to the baby DBAs of the world.

On a side note, before you watch this DVD stick it in your computer and copy the script files.  There are some places that claim the scripts are on the website, but the only thing on the website is a quick note to let you know about the files on the DVD.   I think the scripts should be posted on the website as well as the DVD! We just typed our queries and notes in because we were too lazy to do the whole stop the movie, copy files to everyone, etc. song and dance.  It worked out fine, Kalen doesn’t move too fast and you have time to pay attention and try things out even if you have to type a few lines of code.  Hunt and peck people will have enormous trouble and should definitely copy the scripts beforehand.

It was great fun to geek out DBA style with Kalen’s video, and I really look forward to doing something similar with future videos.

Bookmark and Share

Hey guys, I’m having a screening of Kalen Delaney’s new DVD. You saw the details on my blog, but if you didn’t get a chance, check out http://sqlserverdvd.com for more info.  I’m having the screening tomorrow at 7:30 PM in my house.  I’ve invited about 10 people, if we get a good turnout next time we’ll have it in a bigger place.

 Contact me directly for directions!  wharrislv at gmail dawt com

 Chuck Boyce from SQLServerDVD was cool enough to offer us a copy of the DVD to auction off amongst the attendees.  Come and get your free loot, under the condition that we take care of him next time he comes to Vegas!

Bookmark and Share

I’d love to see some replies to this one, what makes you a geek?

I’m 29 now, got my first network admin job at 17 and first computer at 16. I spent that first year with a computer going crazy reading and learning back when SLIP connections to the net were hard to get!

I had an account on AOL on opening day.   I canceled my Compuserve account to afford it.

I turned down an offer to purchase a Cray supercomputer from a university.  It was bigger than the bedroom I wanted to put it in.

I can draw out the signal path of a mixing board, and have spent enough time recording music to be able to help people create semi professional home recording setups for under $5,000.

I blogged about Microsoft’s proposal to buy Yahoo at 3am, a few hours after Reuters caught it.

I’ve sold 4 different domain names for over $500 a piece.

I’ve run networks for international companies that had voip solutions, thanks to me, as far back as 1996.

My friend and I set up an always on VPN connection so that we could play Rise of Nations over the “LAN” of Cox Cable.

I ran NT 3.51 in a production environment before anyone even considered anything besides mainframes, netware, and unix.

I can make free calls from a large retailer’s phone system here in Las Vegas.

I’ve been heavily into databases for about 6 years, and into MySQL for about 2 years as I’ve been working part time with some crazy web developers. I’m all about design and architechting, standards based SQL programming, and MS SQL administration. Sometimes I write normal sentences in SQL via IM to amuse my developer friends.

I remember when slackware linux was the frontrunner, and I remember when linux was hard to install!

My bookshelves are overflowing with books in general, most of which are about technology related subjects.  You can see the list of books in my bedroom here.

I had a DEC machine in my bathroom, running TCP/IP.  I looked lovingly at it while surfing the Internet from my bathtub.

I’ve modded an xbox with no tools so that my wife could play emulated NES Tetris.

I vote for people based on their stance on technology related issues like Net Neutrality and spectrum availability.  Lessig owns me.

I’m pretty good with linux administration, exchange server, windows server, solaris, and more recently sharepoint and virtualization technologies like vmware.

I recently ordered a machine with 16gb ram for my bedroom, which will include a VMWare server running SSL tunneling to Amsterdam so that I can torrent in peace.

I read the Vista Resource kit from cover to cover for fun.

I know how to program and I never do it.  I’ve learned c, c++, hypercard, pascal, and shell scripting.

I know how easy it is to get into someone’s computer, and how much easier it is to steal an identity.

I know the IT systems in casinos like the back of my hand, I spent 8 years installing and supporting custom software for the gaming industry.  I’ve seen server rooms the size of a football field.

I’ve administered an OS/2 based document management system with an optical storage back end.

I’ve repaired broken CRT monitors with only a soldering iron and a screwdriver.

I had all my music in MP3 form before P2P really existed.

I can get BIOS level remote control to servers in China from my phone.

I actually know how BGP works.

I’ve chaired a special interest group for a local Macintosh user’s group.

I’ve been running IPv6 for years at home.

I’ve been offered to preview and review a technology book in the Head First series.

I’m going to add more as I think of them…what makes you a geek?

Bookmark and Share

Rick Heiges recently mentioned on SQLblog.com a feature that I wasn’t aware of until his mention.  Filtered Indexes are kind of cool!  In a filtered index, you simply add a WHERE clause to your index code and it will only index certain values.

This could be very cool for huge inventory or order tables where you really only ever want the open or current items.  Instead of indexing the entire table and storing all those “closed” or “discontinued” values, you can store a blank value for these and avoid the space taken up for them as well as the sorting necessary to maintain the index with any more than 2 values.  It seems to be something useful for performance and storage, and it apparently has been out for over a decade in products like Fox Pro according to the article.

 If you haven’t checked out SQLblog.com until now, I think you should!

Bookmark and Share

Recently on a private forum that I frequent, a member asked if anyone among us was using server virtualization technology like VMWare in production at their workplace.

My reply is as follows, and should be interesting to anyone considering the viability of server virtualization in small and medium businesses.

We’re using virtualization extensively in production, but ONLY for low usage or administrative functions like Anti Virus reporting, automated software updating, basic file and print services, etc. We’re also using it in production for smaller offices that need to keep hardware costs down while only supporting a few users. We’ve got an entire 10 person network running on a single server running exchange, SQL server for a custom inventory application, and your typical network support software like file and print services, software updating, mobile communications, and threat prevention. We’ve got a spare sitting just below it powered down for emergencies, and the virtualization kept our costs down while allowing us to avoid the irritating software conflicts common to server software.

I think, without a heavy investment into some serious enterprise grade communications and networked storage technology, you can’t get the kind of performance that most of us demand from our SQL servers for a reasonable price. The price to performance ratio is the key to SQL’s widespread virtualization in the future, and it may never get there for small and medium business due to our inability to justify the huge investment in infrastructure that is required.

That being said, we’ve got quite a few virtualized solutions that are using SQL Server in a low utilization scenario (like spam filtering software) and it has worked wonderfully. Likewise for testing environment where virtualization has been an absolute godsend. I think that it all depends on the size of your enterprise and the utilization of the software you’re running when making a judgment on whether or not virtualization is right for you.

Bookmark and Share

Her books are great, her blog posts are super informative, and I admire her very much.  She has a new DVD coming out(which I pre-ordered) that covers some fun SQL Server technology.

 Check out her SQLBlog post here!

Bookmark and Share

Microsoft seems to have pioneered a new method for service packs.  Sharepoint(MOSS) and Exchange 2007 BOTH don’t work right simply applying the service pack.  You need to uninstall, and reinstall using new media/downloads which include both the server package AND the service pack.  When applying service packs in the future, do your research and more importantly do your testing.  The workflow on our MOSS server simply stopped working upon application of SP1, and the answer from Microsoft support was to never run another service pack by itself, but rather to reinstall with the slipstreamed version.  What a pain!

Bookmark and Share

Hopefully, this will get googled, and I’ll find some other DBAs using twitter.

You can find me, here.  It would be great to get a few SQL guys microblogging their thoughts as a group.

https://twitter.com/wharrislv

Bookmark and Share