Google 프로그래밍 방식으로 Java API를 검색하는 방법

Google 프로그래밍 방식으로 Java API를 검색하는 방법

---

프로그래밍 방식으로 Google을 검색 할 수 있는지 여부와 방법을 아는 사람이 있습니까? 특히 Java API가있는 경우 더욱 그렇습니다.

---

몇 가지 사실 :

1. Google은 JSON 을 반환하는 공개 검색 웹 서비스 API를 제공합니다 . http://ajax.googleapis.com/ajax/services/search/web . 여기에 문서화

2. Java는 HTTP 요청을 제공 java.net.URL하고 실행 java.net.URLConnection하고 처리합니다.

3. JSON은 임의의 Java JSON API를 사용하여 Java에서 완전한 Javabean 객체로 변환 될 수 있습니다. 최고 중 하나는 Google Gson 입니다.

이제 수학을하십시오.

public static void main(String[] args) throws Exception {
    String google = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=";
    String search = "stackoverflow";
    String charset = "UTF-8";

    URL url = new URL(google + URLEncoder.encode(search, charset));
    Reader reader = new InputStreamReader(url.openStream(), charset);
    GoogleResults results = new Gson().fromJson(reader, GoogleResults.class);

    // Show title and URL of 1st result.
    System.out.println(results.getResponseData().getResults().get(0).getTitle());
    System.out.println(results.getResponseData().getResults().get(0).getUrl());
}

Google에서 반환하는 가장 중요한 JSON 데이터를 나타내는이 Javabean 클래스를 사용합니다 (실제로 더 많은 데이터를 반환하지만 이에 따라이 Javabean 코드를 확장하는 것은 사용자의 몫입니다).

public class GoogleResults {

    private ResponseData responseData;
    public ResponseData getResponseData() { return responseData; }
    public void setResponseData(ResponseData responseData) { this.responseData = responseData; }
    public String toString() { return "ResponseData[" + responseData + "]"; }

    static class ResponseData {
        private List<Result> results;
        public List<Result> getResults() { return results; }
        public void setResults(List<Result> results) { this.results = results; }
        public String toString() { return "Results[" + results + "]"; }
    }

    static class Result {
        private String url;
        private String title;
        public String getUrl() { return url; }
        public String getTitle() { return title; }
        public void setUrl(String url) { this.url = url; }
        public void setTitle(String title) { this.title = title; }
        public String toString() { return "Result[url:" + url +",title:" + title + "]"; }
    }

}

또한보십시오:

• 다음을 사용하여 HTTP 요청을 실행하고 처리하는 방법 java.net.URLConnection
• JSON을 Java로 변환하는 방법

---

2010 년 11 월 이후 업데이트 (위 답변 2 개월 후), 공개 검색 웹 서비스는 더 이상 사용되지 않습니다 (서비스가 제공되는 마지막 날은 2014 년 9 월 29 일이었습니다). 이제 가장 좋은 방법은 정직한 사용자 에이전트와 함께 http://www.google.com/search를 직접 쿼리 한 다음 HTML 파서를 사용하여 결과를 파싱하는 것 입니다. 사용자 에이전트를 생략하면 403이 반환됩니다. 사용자 에이전트에 누워 웹 브라우저 (예 : Chrome 또는 Firefox)를 시뮬레이션하면 훨씬 더 큰 HTML 응답을 얻을 수있어 대역폭과 성능이 낭비됩니다.

다음은 Jsoup 을 HTML 파서로 사용하는 시작 예제입니다 .

String google = "http://www.google.com/search?q=";
String search = "stackoverflow";
String charset = "UTF-8";
String userAgent = "ExampleBot 1.0 (+http://example.com/bot)"; // Change this to your company's name and bot homepage!

Elements links = Jsoup.connect(google + URLEncoder.encode(search, charset)).userAgent(userAgent).get().select(".g>.r>a");

for (Element link : links) {
    String title = link.text();
    String url = link.absUrl("href"); // Google returns URLs in format "http://www.google.com/url?q=<url>&sa=U&ei=<someKey>".
    url = URLDecoder.decode(url.substring(url.indexOf('=') + 1, url.indexOf('&')), "UTF-8");

    if (!url.startsWith("http")) {
        continue; // Ads/news/etc.
    }

    System.out.println("Title: " + title);
    System.out.println("URL: " + url);
}

---

In the Terms of Service of google we can read:

5.3 You agree not to access (or attempt to access) any of the Services by any means other than through the interface that is provided by Google, unless you have been specifically allowed to do so in a separate agreement with Google. You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services.

So I guess the answer is No. More over the SOAP API is no longer available

---

To search google using API you should use Google Custom Search, scraping web page is not allowed

In java you can use CustomSearch API Client Library for Java

The maven dependency is:

<dependency>
    <groupId>com.google.apis</groupId>
    <artifactId>google-api-services-customsearch</artifactId>
    <version>v1-rev57-1.23.0</version>
</dependency> 

Example code searching using Google CustomSearch API Client Library

public static void main(String[] args) throws GeneralSecurityException, IOException {

    String searchQuery = "test"; //The query to search
    String cx = "002845322276752338984:vxqzfa86nqc"; //Your search engine

    //Instance Customsearch
    Customsearch cs = new Customsearch.Builder(GoogleNetHttpTransport.newTrustedTransport(), JacksonFactory.getDefaultInstance(), null) 
                   .setApplicationName("MyApplication") 
                   .setGoogleClientRequestInitializer(new CustomsearchRequestInitializer("your api key")) 
                   .build();

    //Set search parameter
    Customsearch.Cse.List list = cs.cse().list(searchQuery).setCx(cx); 

    //Execute search
    Search result = list.execute();
    if (result.getItems()!=null){
        for (Result ri : result.getItems()) {
            //Get title, link, body etc. from search
            System.out.println(ri.getTitle() + ", " + ri.getLink());
        }
    }

}

As you can see you will need to request an api key and setup an own search engine id, cx.

Note that you can search the whole web by selecting "Search entire web" on basic tab settings during setup of cx, but results will not be exactly the same as a normal browser google search.

Currently (date of answer) you get 100 api calls per day for free, then google like to share your profit.

---

Indeed there is an API to search google programmatically. The API is called google custom search. For using this API, you will need an Google Developer API key and a cx key. A simple procedure for accessing google search from java program is explained in my blog http://preciselyconcise.com/apis_and_installations/search_google_programmatically.php

---

Google TOS have been relaxed a bit in April 2014. Now it states:

"Don’t misuse our Services. For example, don’t interfere with our Services or try to access them using a method other than the interface and the instructions that we provide."

So the passage about "automated means" and scripts is gone now. It evidently still is not the desired (by google) way of accessing their services, but I think it is now formally open to interpretation of what exactly an "interface" is and whether it makes any difference as of how exactly returned HTML is processed (rendered or parsed). Anyhow, I have written a Java convenience library and it is up to you to decide whether to use it or not:

https://github.com/afedulov/google-web-search

---

As an alternative to BalusC answer as it has been deprecated and you have to use proxies, you can use this package. Code sample:

Map<String, String> parameter = new HashMap<>();
parameter.put("q", "Coffee");
parameter.put("location", "Portland");
GoogleSearchResults serp = new GoogleSearchResults(parameter);

JsonObject data = serp.getJson();
JsonArray results = (JsonArray) data.get("organic_results");
JsonObject first_result = results.get(0).getAsJsonObject();
System.out.println("first coffee: " + first_result.get("title").getAsString());

Library on GitHub

---

In light of those TOS alterations last year we built an API that gives access to Google's search. It was for our own use only but after some requests we decided to open it up. We're planning to add additional search engines in the future!

Should anyone be looking for an easy way to implement / acquire search results you are free to sign up and give the REST API a try: https://searchapi.io

It returns JSON results and should be easy enough to implement with the detailed docs.

It's a shame that Bing and Yahoo are miles ahead on Google in this regard. Their APIs aren't cheap, but at least available.

---

Just an alternative. Searching google results can also be done using any HTML Parser such as Jsoup in Java. Following is the link to example.

https://www.codeforeach.com/java/example-how-to-search-google-using-java

참고URL : https://stackoverflow.com/questions/3727662/how-can-you-search-google-programmatically-java-api