publicstaticStringgetHtmlCode(Stringurl,StringcharCode)throwsIOException{StringhtmlCode="";InputStreamin=null;HttpURLConnectionconnection=null;try{URLurlCon=newURL(url);c...
public static String getHtmlCode(String url,String charCode) throws IOException {
String htmlCode = "";
InputStream in = null;
HttpURLConnection connection=null;
try {
URL urlCon = new URL(url);
connection = (HttpURLConnection) urlCon.openConnection();
connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.25 Safari/534.24");
connection.setConnectTimeout(5000);
connection.setReadTimeout(5000);
connection.setFollowRedirects(true);
connection.setInstanceFollowRedirects(false);
connection.setDefaultUseCaches(false);
in = urlCon.openStream();
int index = 0;
byte[] bytes = new byte[1024*100];
int count = in.read(bytes, index, 1024 * 100);
while (count != -1) {
index += count;
count = in.read(bytes, index, 1);
}
htmlCode = new String(bytes, charCode);
} catch (Exception e) {
e.printStackTrace();
}
finally{
if(in!=null)
in.close();
if(connection!=null)
connection.disconnect();
}
return htmlCode;
}
我要抓取一个页面的数据,但是因为访问的页面采用AJAX。使我读取的数据不完全。请问下该怎么写才能读取到全部的数据
展开