C# Web Scraping Library

I just started to delve into website scraping as I periodically like to hit a website for current prices and fluctuations on a timely basis.

I believe I was listening to a podcast of .NET Rocks – Scrappy where they mentioned this library and if you know what episode please tell me so I can update the post for others who want to listen.

It really looks simple to use and pretty great so I’m currently playing with it now and I’ll write more about this as I use it and what project I decide to use it on.


What software do I have loaded on my Windows machine?

When I get a new windows machine I always load up a specific set of software. I always start with these two projects to help me get them loaded:

  1. Ninite – Install and Update All Your Programs at Once
  2. Chocolatey – Software Management Automation

I use Ninite first as it has these basic apps that I love to load up all the time:

  • Chrome
  • Opera
  • Firefox
  • Dropbox
  • Google Drive
  • OneDrive
  • FileZilla
  • Notepad++
  • Visual Studio Code
  • Skype
  • Evernote
  • 7-Zip
  • WinRAR
  • .NET 4.6.2
  • Essentials
  • TeamViewer 12
  • WinDirStat
  • More to come…

After I have installed all of the Ninite software, I then get Chocolatey setup and use the command line to install these apps:

  • Git
  • Node JS
  • More to come…

I do also install a few development tools and other misc. file seperately that don’t come with the packages above:

  • Visual Studio Community 2015
  • SQL Server Express & Management Studio
  • More to come…

not quite finished yet…

Data Annotations Examples for C#

Using data annotations in c# is a simple way to handle validations in your model. The example below show you how to:

  1. Make a filed “required”
  2. Change the “display name” for what the clients see
  3. Denotes the field as an “email” type
[Display(Name = "Email Address")]
public string Email { get; set; }

Here’s an example of “string length” with an “error message”:

[StringLength(100, ErrorMessage = "The {0} must be at least {2} characters long.", MinimumLength = 6)]
public string Title { get; set; }

If you’re working with a password field and want to also confirm the password, here’s an example of that:

[StringLength(100, ErrorMessage = "The {0} must be at least {2} characters long.", MinimumLength = 6)]
[Display(Name = "Password")]
public string Password { get; set; }
[Display(Name = "Confirm password")]
[Compare("Password", ErrorMessage = "The password and confirmation password do not match.")]
public string ConfirmPassword { get; set; }

If you’re dealing with a range where you want the user to pick from 0 to 10 per se, you want to use the “Range” data annotation. Below is an example of using credits with 0 to 5:

[Range(0, 5)]
public int Credits { get; set; }

If you want to create a textarea which has several lines for users to type in, this is the annotation you can use:

public string Body { get; set; }

not quite finished yet…

Passive income rules to live by

What’s passive income?

It’s money that you have coming in that doesn’t involve you having to i.e. spend an hour of your time to receive an hour worth of pay.

Very clear passive income earnings that most people understand is a “rental property”. You receive money from it and you don’t even live there.

So along those same lines, you can look at a software app that does a similar thing, i.e. you build a website, you put advertisements on it and it makes money for you as long as you have traffic coming to it.

This post isn’t about learning what “passive income” is, but a couple of rules that you should always remember when picking out your new business idea are:

  1. Have  “Automated Sales”
  2. Have “Recurring Revenue”
  3. Keep your “Accounts Payable” to an absolute minimum i.e. Get to be as “debt free” as you can as a daily goal
  4. No “Account Receivables”
  5. Look for ideas where you “Sell Tools” or “Utilities” i.e. if you lived in the age of the “Gold Rush”, selling shovels made you a wealthy person
  6. Do as much as you can as a “Soloprenuer” because the moment you take on a partner, your profits split in two

Questions to ask when researching new business opportunities

When you’re looking for new business ideas, it’s always great if you can go out and interview some actual people who would be using your product.

I believe I got these questions from Dane Maxwell who talks about how to start a business as his mantra.

The idea for brainstorming new business ideas it to “Find The Pain”!

Once you find the pain of any business, it’s a money making opportunity. You should look to be “Cheaper, Simpler, Smaller and More Convenient” than the rest which is a quote attributed to Clayton Christensen from his book The Innovators’s Dilemma.

Here’s a couple good ones to ask as you interview the people who would be users of your new product:

  1. What is the most important activity in your business?
  2. Is there any pain associated with that activity?
  3. What problems cost you the most money?
  4. What is the most painful part of your day?