PHP is what's known as an interpreted language. It's not compiled into a program that can be run, and the only way that it can be run is by a server that can read PHP.
PHP is server parsed, meaning that when you view a PHP file, you see the output of the PHP file, not the actual PHP code. It's known as server parsed because the server runs the PHP script and then gives your web browser the output in a format it understands, like HTML. Because your web browser doesn't understand PHP, you need to run a web server to be able to test the scripts you write. I recommend Apache Web Server, the free, easy to use web server, available for Unix, Linux and even Windows 95/98/NT. Apache is available from www.apache.org.
Before you install Apache, you must make sure that your computer has a server name. On Windows you set this by going to Network Neighborhood Properties, and editing the TCP/IP properties. Just call it something like mycomputer, and put the domain as local. Under Redhat Linux (my experience is limited with things apart from Redhat, it may be the same) use linuxconf to set the ServerName and Domain.
Now that you have a web server running, you need to get PHP from www.php.net. Luckily, PHP is one of the best documented languages I've ever seen. Unless you have a permanent internet connection, download the manual, it's an invaluable resource. Follow the installation instructions in the PHP manual, and your computer can read PHP! Now that you have your development enviroment working, it's simple. Just use any old text editor, save you files with the .php extension into the Apache web page directory (named htdocs under whereever you installed Apache). This tells your server that you want it to interpret this document as PHP.
Start off by just trying some of the examples in the PHP manual. When you want to view them, start Apache (Windows: Start->Programs->Apache Group->Start Apache; UNIX: Type "apache" from the console), then open your web browser, and type in either:

• http://localhost/nameofdocument.php
• http://mycomputer.local/nameofdocument.php
• Or your computers IP address/nameofdocument.php