Logo 
Search:

Sharepoint Forum

Ask Question   UnAnswered
Home » Forum » Sharepoint       RSS Feeds

Sharepoint help needed

  Asked By: Noah    Date: Jan 01    Category: Sharepoint    Views: 1882

Sharepoint V1, SP2 and SP1
Does anyone have technical information on how a content
crawl takes place as far as permission
challenge/response? Also, beyond just read permissions
to the content source, what exact permissions does an
crawl account need?

We have a problem whereby secure local intranet sites are
not being crawled. The access account does have read
permissions to the local intranet sites but we keep on
getting an access denied error. We can access the site
with IE using the crawl account just fine. I have set up site path rules with
the account and this does not help either.

In fact,I have further tested this domain account and made it a member of the
local admin group on the sps server as well as the web server I am trying to
crawl. I have setup the account in the site path rules. Also, this account is
a coordinator on the workspace. Unsecured parts of the site crawl work fine but
secure parts get the access denied error even when the crawl account has full
NTFS permissions to the site.

Share: 

 

11 Answers Found

 
Answer #1    Answered By: Otis Blackwell     Answered On: Jan 01

Just a few thoughts:

- Are you trying to access  the crawl  site with IE from the server
itself?

- Have you tried putting in a domain  as the username? ie
DOMAIN\username instead of just username

- Are these two systems in the same domain?

- does the content  have a robots.txt file or any meta tags that show
that it shouldn't be crawled?

- Have you checked the content access logs?

- You mention NTFS, but you also mention IE. What kind of content
source are you crawling, a file share (ie \\server\share) or a web  site
(http://server/)?

To my knowledge, you just need read  access. Check out Bill English's
book for a lot of information  and step-by-step instructions to access
restricted content.

 
Answer #2    Answered By: Virendar Chaudhari     Answered On: Jan 01

I am crawling the site  using a content  crawl not IE.

I have tried username\domain

The systems are in the same domain.

There are no metatags or robots.txt file.

Yes I have checked the content access  logs and the site IIS logs. The site IIS
log on the web server  shows an access denied  error 401. The interesting thing is
that the account  it says is passing credentials is the anonymous account of the
web server not the crawl  account. It seems that the crawl process is not passing
its credentials to the secure  site. On the SharePoint side, I get the access
denied error  on the log... Error fetching URL, (80041205 - Access is denied.
Make sure the account used to access this URL is the correct account. )

The content I am trying to crawl is a web  site.

 
Answer #3    Answered By: Mitchel Villarreal     Answered On: Jan 01

By chance is the web site  you are trying to crawl  ( the site that is
returning the 401) have Enable Anonymous Access checked?

 
Answer #4    Answered By: Ranu Badhan     Answered On: Jan 01

Yes it does, and certain parts  of the site  are secure  requiring logon. The
content crawl account  does have logon rights.

 
Answer #5    Answered By: Irene Moss     Answered On: Jan 01

Any chance you can remove the anonymous access  check and start up an
index on the site  to see if you stop getting errors?

 
Answer #6    Answered By: John Scott     Answered On: Jan 01

Are the site  path rules  specific to the areas that require logon, or
generic for the entire site?

I don't have any web sites  that require authentication in one area but
not in others, so I don't have anything to test.

 
Answer #7    Answered By: Donald Torres     Answered On: Jan 01

Yes I setup  a specific site  path rule for the secure  site and then also created
crawl  that only crawls the secure site.

 
Answer #8    Answered By: Courtney Scott     Answered On: Jan 01

have you solved your problem  yet?

 
Answer #9    Answered By: Jagdeep Hor     Answered On: Jan 01

nope the problem  has not been solved. As a matter of fact  I have security
loggind enabled for the folder on the web site  as well as monitoring the
packets. It seems that the crawl  process is not using the access  account
speicified in the site pathc rules. From looking at the packets there is no
challenge response provided by the crawl process. Any ideas?

 
Answer #10    Answered By: Aja Howe     Answered On: Jan 01

What sticks in my head is the allow anonymous access  for the web  site.
If Allow Anonymous access is checked then even if windows Auth or basic
auth is checked, the server  defaults to anonymous account  and never
bothers to check to see who is asking. Now , if you request a file that
needs more access then the anonymous, iis is supposed to tell you that
you do not have permissions  and then bounce back with a response letting
you know what type of authentication is allowed which in IE you will see
a pop up for Basic, or it will get the credentials from the server if
Windows Auth is allowed.

So.... I wonder if SPS goes out to crawl  your secured site  passing the
access account you specified, yet since you are allowing Anonymous
access, the credentials gets effectively thrown out and access via
Anonymous. Then the page requires a specific account and sends a not
authorized header back and SPS does not resend the credentials.

This is why I wondered if you could try the index with the Allow
Anonymous Access off, and see if your content  access account is sent
correctly.

I am not a iis expert, but I have had issues with this Anonymous access
handling before ...

 
Answer #11    Answered By: Cecil Mckenzie     Answered On: Jan 01

I unchecked the setting for anonymous but this still does not work.


What sticks in my head is the allow anonymous access  for the web  site.

If Allow Anonymous access is checked then even if windows Auth or basic

auth is checked, the server  defaults to anonymous account  and never

bothers to check to see who is asking. [If anonymous is checked IIS will try to
serve the file using the anonymous account. If the anonymous account does not
have the proper NTFS permissions  to the file IIS will not serve the file. If
anonymous and either of the other authentication method boxes are checked IIS
will still try to serve the file using anonymous. If the anonymous account does
not have the proper NTFS permissions to the file then IIS will let you know what
type of authentication method it will allow, and the user will see a pop up for
some sort of authentication. IE, when challenged, can also silently pass
credentials to IIS if configured to do so.] Now , if you request a file that

needs more access then the anonymous, iis is supposed to tell you that

you do not have permissions and then bounce back with a response letting

you know what type of authentication is allowed which in IE you will see

a pop up for Basic, or it will get the credentials from the server [IE can be
configured to silently pass user credentials when challenged. IIS needs some
sort of a response from the user in order to authenticate.] if

Windows Auth is allowed.

 
Didn't find what you were looking for? Find more on Sharepoint help needed Or get search suggestion and latest updates.




Tagged: