Network Options Setup

The Surf3D Network Options include the ability to confine the search to simple URL types, bypass networked cached pages to get timely news, disable cookies, set directory search depth, and set web page referrer information.

N icon on the main display Toolbar: Network Options control panel for basic online preferences.

Network Options: Click on the –Sharp N- icon. This opens the Network Options control panel. You can leave them for now, if you started with demo2.XO as your starting point template for creating a new agent.

Search didn’t do anything?If the pulsing colored box in the top left corner of the main toolbar window immediately turns to red and the search doesn’t execute, you may need to change the Network Options settings.

Click on the –N- icon to open the panel and click on the "Include Text Based URLs" to enable grabbing URLs that are embedded in any plain text document (.TXT).

Also make sure the Maximum Subdirectory is set to All Subdirectories (no limit).

Now click on Accept, and try running the agent module again by clicking the -Right arrow- icon. It should run.

If it does not, then go back to the Network Options panel and click on the Force Origin Server Checkbox, and try running the agent module again.

In a few cases, the web site may be served from another domain than the domain you started with in the Search window GET URL. In that case, open the Search panel (-X- icon on main toolbar) and un-check the "Stay in Domain" box, and try running the agent again.

Make sure you have not checked the Nav checkbox or any other checkbox in the search set up control panel.

If the agent module still fails to run, the web site may be all Java applet driven or complex Javascript, or a complex Flash site interface. If the site offers it, switch to non-Flash interface on your browser window and click the GET URL again.


What are the different Network Options Operations settings, and what do they each mean, and when do I need to use them?

Confine Search to Basic URL Names – there are several distinctions on URLs, and when this is checked, Surf3D only looks at simple addresses, such as .html, .asp and a couple other common URL suffixes. Anything at eBay is typically not a simple URL.

Use Cache if the Network Fails – this checkbox will instruct the agent to use your browser cache if you are offline and the data exists in your cache from earlier manual surfing you have don with the browser, or earlier agent runs that have stored items in the browser cache.

Force Origin Server - selecting this checkbox will cause any network web proxies to not be used, and instead get the page from the original source and not the network cache that is currently in use on the network. This can be important for retrieving timely information given network caches pages can be sometimes a little dated.

Force Reload and Add to Cache – selecting this checkbox causes the agent module to not use previously existing data and files stored in your browser cache and instead only get the page from online. This is like the reload button on your browser.

Disable Cookies – selecting this checkbox prevents any cookies from being deposited into your temporary internet files, which might be encountered during an agent search.

Include Text Based URL’s – selecting this checkbox lets the agent module collect http URL links in Javascript text in many cases, but is designed for grabbing URL’s from text documents. For example this is must be checked if you have created a stock portfolio with URLs for Yahoo quotes for every stock, and stored the URLs in alist in the agent text doc file (accessed via clicking on the -?- icon on the agent module main toolbar (e.g. see: NSDQtop50Agent_1.xo).

What is the Maximum Subdirectory limit mean in the Network Options panel, and when should I set it to other than "no limit"?

This set of radio buttons controls the depth of search relative to a given web site domain.

If set at Top Only – No Subdirectories, only the single page will be visited.

If set at –1-, only one level down from the top directory will be visited.
e.g. domainname.com/subdirectory

If set at –2-, only 2 levels down from the top directory will be visited.
e.g. domainname.com/subdirectory/subdirectory

If set at –3-, only 3 levels down from the top directory will be visited.
e.g. domainname.com/subdirectory/subdirectory/subdirectory

If set at All Subdirectories (no limit) then Surf3D will not be limited.

What is the Initial Referrer line in the Network Options panel?

When you go to a web page, if you have been to another web page just before it, then the last web page you visited can be identified by the newly visited web page server. If you don’t mind this (as this is how browsers normally work anyway), leave the text NONE in the entry line. However, if you want to identify a particular referring web page to the web page or pages your Surf3D agent module visits, you can type in URL here. This effectively spoofs the referral info to the next server surf3D is visiting. We recommend however you leave this as NONE, unless you are concerned with maintaining extra privacy. Sometimes web pages will only load if the referring page is a particular one, and so this can be a convenient way to introduce the expected referring page address so you can have access to the page.

What is the I.D. line mean in the Network Options panel?

This identifies who you are to the servers that Surf3D is visiting. Surf3D looks to any server as if it is merely a browser, since the I.D. is Mozilla/4.0 (compatible; MSIE 4.01; Windows 95). Of course you can type in your browser type if you want.

Copyright © 2001-2008 Navagent, Inc.