In your Theme Settings which Category have you selected as the Home Category?

I have tried this, which works, but it still shows all the posts from the other category?

http://windlestrawquilts.co.uk/news/

In your Menu Manager, rather than having a page designated for Blog, setup a Category as a page. So, in the Menu Manager, add the Category you want to display to the Menu, assign posts to that Category, and you should be good :)

How would I exclude a certain category from the blog page? At the moment it's showing ALL posts, but I only require 1 category to be displayed on the blog.