We just got rid of our as400  but for the last few years I've been using
ADO and ADO.NET to put data in and get data out of it. I used the ODBC
provider with the I-Series Client Access Drivers to get data inside of a
Web Part. You'll have to have the IBM I-Series Client Access drivers
installed on the Sharepoint server running the Web Part.

As for the IBM OLE DB provider, I only tried to use it from Integration
Services and had a lot of trouble with it (though I assume if you can
get it to work that it will perform much better than it's ODBC
equivelant). Here's an example connection string I used:

DRIVER=Client Access ODBC Driver
(32-bit);UID=YourUsername;PWD=YourPassword;PREFETCH=1;SORTTYPE=1;DESC=iS
eries Access for Windows ODBC data
source;PKG=QGPL/DEFAULT(IBM),2,0,1,0,512;LANGUAGEID=ENU;DFTPKGLIB=QGPL;D
BQ=YourDatabaseCatalog;SYSTEM=as400.yourcompany.com;