php forum
php mysql forum
php mysql smarty
 
Page 1 of 2 1 2 >
Topic Options
#227647 - 11/10/02 09:04 PM data retrieval - public domain
fishtails Offline
Enthusiast

Registered: 06/19/02
Posts: 307
Loc: Fernandina Beach, FL
I'm working on a script which pulls tide data from NOAA, parses it and the inserts into my PAL page. You can see it working at http://www.fish-tails.net<br /><br />Anyway, I need a little help on regexp, and maybe some suggestions.<br /><br />Ideally, I would like to pull the data from NOAA, strip html completely out, and then insert the data into mysql. Once the data is stored, I can then write a script so that the user can select the reference stations they want to be displayed on the home page, or in their My Home.<br /><br />Currently, I have it loading directly to the home page without stripping html. Here is the code I'm using.<br /><br /> <pre><font class="small">code:</font><hr> <br />$month = date("n");<br />$day = date("j");<br />$year = date("Y");<br />$next = (date("j")+1);<br />$next1 = $next+1;<br />$next2 = $next1+1;<br />$next3 = $next2+1;<br />$tide1 = "$month/$day/$year";<br />$tide2 = "$month/$next/$year";<br />$tide3 = "$month/$next1/$year";<br />$tide4 = "$month/$next2/$year";<br />$tide5 = "$month/$next3/$year";<br /><br />//$url = 'http://www.co-ops.nos.noaa.gov/tides/seFB.html';<br />$url = 'http://co-ops.nos.noaa.gov/tides/get_pred.shtml?stn=0030+Fernandina+Beach&secstn=Simpson+Creek,<br />+A1A+highway+bridge&thh=+0&thm=05&tlh=+0&tlm=1<br />9&hh=*0.84&hl=*0.65';<br /><br />$lines_array = file($url);<br /><br />$lines_string = implode('', $lines_array);<br />eregi("$tide1(.*)$tide5", $lines_string, $head);<br />//eregi("<BODY BGCOLOR=\"#FFFFFF\">(.*)</body>", $lines_string, $head);<br />//eregi("<pre>(.*)</pre>", $lines_string, $head);<br /><br />//$head = eregi_replace("</tr><tr><td align=\"right\" width=\"60\">$tide5</td></tr>", "</tr>", $head);<br /><br /><br /><br />echo "I'm running some tests now on the tides program I'm writing. Please use these with discretion. Thanks. DATA from NOAA";<br />echo "<br><b>Simpson Creek, A1A highway bridge 2002 Water Level Predictions</b>";<br />echo "<br>(Reference station - Fernandina Beach)";<br />echo "<br>Corrections - Time Diff. (High 0 hr. 05 min. Low 0 hr. 19 min.)";<br />echo "<br>Hgt. Diff. (High *0.84 Low *0.65)";<br />echo "<br>All times listed are in Local Time, and all heights are in Feet referenced to Mean Lower Low Water (MLLW).";<br />echo "<br><hr><table><tr>";<br />echo "<th width=\"60\">Date</th><th width=\"60\">Time</th><th></th><th width=\"40\">Ht.</th><th>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br /></th><th>Time</th><th></th><th width=\"40<br />\">Ht.</th><th>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br /></th><th>Time</th><th></th><th width=\"40<br />\">Ht.</th><th>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<br /></th><th>Time</th><th></th><th width=\"40<br />\">Ht.</th></tr><tr><td>";<br /><br />echo "<font face='arial'>";<br /><br /><br />echo $head[0];<br /><br /><br />echo "</td></tr></table>";<br /> </pre><hr> <br /><br />You will note I have two $url with one commented out. The one that is commented is the one I think woul be the easiest to work with, but then I'm not sure, which is why I'm here. <img src="/forum/images/graemlins/wink.gif" alt="" /><br /><br />The second one is the one that I'm using on my home page. Now, first off, I only want 4 days displayed, but if you look at my home page a new line with a fifth day starts. I've tried preg_replace and ereg_replace but to no avail. Anyone have a good understanding of regexp that can help me out?<br /><br />Regarding the database. Looking for suggestions here. Should I set up a cron (I have an idea what it does but have never used it) to pull the data I want nightly, insert it into a db as one record, then have my pal box pull from that record? I would think that way there would be faster page load times. I'm open for suggestions here.<br /><br />Any help is appreciated.
_________________________
Fish or Cut Bait! http://www.fish-tails.net

[:"red"]Air traffic control? It has its ups and downs.[/] http://www.natcazjx.org

Top
#227648 - 02/03/03 06:36 AM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Did you make any headway on this? I would like to do something similar in my neck of the woods. (Washington)

Top
#227649 - 02/03/03 03:33 PM Re: data retrieval - public domain [Re: Kelly]
fishtails Offline
Enthusiast

Registered: 06/19/02
Posts: 307
Loc: Fernandina Beach, FL
I'm still working on it. You can check out what I have so far here. http://www.fish-tails.net/threads/tidecal.php
_________________________
Fish or Cut Bait! http://www.fish-tails.net

[:"red"]Air traffic control? It has its ups and downs.[/] http://www.natcazjx.org

Top
#227650 - 02/03/03 04:10 PM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Very nice.. Is this cached? or does it draw on their server every time?

Top
#227651 - 02/03/03 04:14 PM Re: data retrieval - public domain [Re: Kelly]
dimopoulos Offline
Kahuna

Registered: 08/18/02
Posts: 1271
Loc: Vienna, Austria
Nice!!!<br /><br />Nikos
_________________________
Nikos

Top
#227652 - 02/03/03 04:25 PM Re: data retrieval - public domain [Re: Kelly]
fishtails Offline
Enthusiast

Registered: 06/19/02
Posts: 307
Loc: Fernandina Beach, FL
Thanks. For the moment it draws on their server each request. I have to learn more regarding cache. Originally I wanted to get the data and insert into my db, but I haven't gotten that far yet. The best approach would be, I think, to cache it. Any help in this area is appreciated.
_________________________
Fish or Cut Bait! http://www.fish-tails.net

[:"red"]Air traffic control? It has its ups and downs.[/] http://www.natcazjx.org

Top
#227653 - 02/04/03 04:50 AM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
I use to use some commercial scripts that would strip and cache the sites I wanted. These worked great until they stopped maintaining them. It was in perl and the code was garbled, they used regex and formatting to get them squared away.<br />It is much nicer to see the open code so adjustments can be made. My php skill are beginner level still... was focused on learning the perl language but moving to php I think is a better option.<br />I hope to help in the future as I can get up to speed... Will kep my eyes open for some php cache scripts we can use on this sort of thing. Might be something over at the php project site.

Top
#227654 - 02/06/03 08:54 PM Re: data retrieval - public domain [Re: Kelly]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
I see the iip integration is using cache. Might be some useful to look at how its used in that script.

Top
#227655 - 02/07/03 12:57 AM Re: data retrieval - public domain [Re: Kelly]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
lol I tried putting your text to work... didn't fair so well<br /><br />http://www.steelheader.net/tide/tidecal.php

Top
#227656 - 02/07/03 11:37 AM Re: data retrieval - public domain [Re: Kelly]
fishtails Offline
Enthusiast

Registered: 06/19/02
Posts: 307
Loc: Fernandina Beach, FL
That code doesn't work. Try my Tide Sticker! Demo at http://www.fish-tails.net in one of the custom pal boxes.


Attachments
67943-tidesticker.txt (270 downloads)

_________________________
Fish or Cut Bait! http://www.fish-tails.net

[:"red"]Air traffic control? It has its ups and downs.[/] http://www.natcazjx.org

Top
#227657 - 02/07/03 07:09 PM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
got it working!!! Nice job.

Top
#227658 - 02/07/03 11:43 PM Re: data retrieval - public domain [Re: Kelly]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
I ran into trouble putting this into a pal box. Got a parse error right where code begins. <br /><br />Should I be including it some other way? A ssi?<br /><br />Works perfect when I call it from a url. Sure its just me but thought I would check.<br />Thanks

Top
#227659 - 02/08/03 11:20 AM Re: data retrieval - public domain [Re: Kelly]
fishtails Offline
Enthusiast

Registered: 06/19/02
Posts: 307
Loc: Fernandina Beach, FL
All you need to do is add this to your custom pal box.<br /><br />include ("path/to/tidesticker.php");
_________________________
Fish or Cut Bait! http://www.fish-tails.net

[:"red"]Air traffic control? It has its ups and downs.[/] http://www.natcazjx.org

Top
#227660 - 02/09/03 02:45 AM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Thanks will get to it tomorrow... fish call me <img src="/forum/images/graemlins/smile.gif" alt="" />

Top
#227661 - 02/09/03 07:17 AM Re: data retrieval - public domain [Re: Kelly]
fishtails Offline
Enthusiast

Registered: 06/19/02
Posts: 307
Loc: Fernandina Beach, FL
I know what you mean! <img src="/forum/images/graemlins/grin.gif" alt="" /> After all, we do have fishing sites that need reports. <img src="/forum/images/graemlins/wink.gif" alt="" /> I fished out of a kayak for the first time the other day. A challenging experience.<br /><br /> []http://www.fish-tails.net/photopost/data/3000/2img_0014-med.jpg[/]
_________________________
Fish or Cut Bait! http://www.fish-tails.net

[:"red"]Air traffic control? It has its ups and downs.[/] http://www.natcazjx.org

Top
#227662 - 02/10/03 12:20 AM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Looks like a good time. I always feel like I am going to tip over in one of those.

Top
#227663 - 02/11/03 08:27 PM Re: data retrieval - public domain [Re: ]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Got it up and running with some additional tides showings.<br /><br />http://www.steelheader.net/ubbthreads/index.php

Top
#227664 - 02/12/03 04:45 AM Re: data retrieval - public domain [Re: Kelly]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Seems to be fairly slow loads since I added the extra tides.<br />Searched for a cache script we could use on it.<br />Benedetto Patrice code posted on webdev<br />
Code:
 // First create a folder named "cache"  where the files will be written <br />// define the fonction <br />function BZ_cache ($url,$cacheName) <br />{ <br />global $cache ; <br />global $QUERY_STRING ; <br />// You may use the naming convention of you choice here <br />$filename = "cache/".$cacheName.",".$QUERY_STRING.".html" ; <br /><br />// You may use test the date of the cache  here <br />    if ( file_exists (   $filename) ) <br />       {  <br />       readfile ($filename) ; <br />       return 1 ; ///  Ok i'v send the html page   <br />       } <br />else <br />      { <br />     if ( ! isset ($cache ) ) <br />        { <br />         $fcontents = join ('', file ($url."?".$QUERY_STRING."&amp;cache=t")); <br />         $fp = fopen ($filename , "w"); <br />         fwrite ($fp, $fcontents  ); <br />         fclose ( $fp) ; <br />            return 0 ;   ///  i'v to execute the file <br />         } <br />      } <br />} <br />// then put this line at the TOP of you file ( before any include ) <br />//  if ( BZ_cache ("COMPLETE URL WITHOUT PARAMS" , "AN_IDENTIFIER" ) )   exit () ; <br /><br /><br />// Example in the file http://azerclic.labynet.org/doc.php3 <br />// the first line is <br />// if ( BZ_cache ("http://azerclic.labynet.org/doc.php3" , "doc" ) )   exit () ; <br />// you will see  the cached files in the  folder http://azerclic.labynet.org/cache/ <br />// That's all <br /> 

Top
#227665 - 02/25/03 04:28 AM Re: data retrieval - public domain [Re: Kelly]
smilesforu Offline
Junior Member

Registered: 01/31/02
Posts: 676
Just did a cache with a cron job on a tide script puller. Had to do a rewrite with help from one of my members to add other tides for the pal. <br /> <br />I have the script pulling the data witha regex and saving it to a text file.... then I use a include on the text file. Set the cron up to run the script so it keeps the text file updated. Loads like a dream... Thanks for the inspiration to get me over the hump on this. Using this same style of a feature to fetch the latest news and regulations, river levels.... next is weather. <br />Very close to having this setup for a mysql database, with cron for river levels. It will also have user login cookie so users can save there tides/river levels of choice. <br /> <br />BTW I used Fastcron.com for remote pull on the scripts since it was cheaper and very easy to setup. Cron job is the way to go!! <br /> <br />Not sure whether to include this with language files or not.. but probably will once its done. <br /> <br />Any day one a couple minor bugs are worked out... <img src="/forum/images/graemlins/smile.gif" alt="" /> <br /> <br />Now back to your regular scheduled fishing trip []http://www.steelheader.net/ubbthreads/images/graemlins/jumping%20king.GIF[/] <br /> <br /> <br />


Edited by Smilesforu (02/25/03 04:43 AM)

Top
#227666 - 03/01/03 12:10 AM Re: data retrieval - public domain [Re: Kelly]
chillin Offline
Journeyman

Registered: 01/26/03
Posts: 127
Loc: bend, OR USA
very awesome guys, i think i'll hack this out for use with mountain/avalanche forecasts from NOAA as well, this will be a wonderful feature.
_________________________
i see threads people
http://www.chinooktc.com

Top
Page 1 of 2 1 2 >



Latest Posts
[7.2.1] - Naked shoutbox
by bellaonline
05/05/12 05:00 PM
[7.x] Stop Forum Spam Integration v0.4
by bellaonline
05/05/12 03:53 PM
Shout Box

(Views)Popular Topics
Known public proxy servers 1689885
Integrated Index Page (IIP) 5.3.1 555705
Finished-[6.5.2] Games Arcade Deluxe v1.9 501236
Integrated Index Page (IIP) 5.1.1 415112
TLD Bv2.1 Released - Threads Links Directory 396822
[6.0x] Who's Online 4.0.0 [Finished] 389412
Finished-[6.5.1] Integrated Index Page (IIP) 6.5 330423
Q & A 298663
Slash UBB 266936
[6.3.x] [beta] Hit Hack 2.0 227970
Forum Stats
13621 Members
59 Forums
37191 Topics
295716 Posts

Max Online: 686 @ 06/28/07 07:04 AM

 

 

 
fusionbb message board php hacks