When writing a paper in LaTeX, you may need to convert citations to BibTeX format. If you have many references, this repetitive work really isn't worth doing. If you already use a reference-management tool such as EndNote or Zotero, you can export them all in one click. Otherwise, this article provides a solution.
Solution: Crossref API + Google Scholar API
Crossref is the largest DOI platform for foreign-language publications and contains metadata for virtually all of them. However, some works—including arXiv papers, among others—can't be found there, so you'll need Google Scholar's help.
To save you time, I've wrapped both APIs. All you need to do is install the package with pip:
pip install get_bibtex
You can then use it as follows:
from apiModels.get_bibtex_from_crossref import GetBibTex
from apiModels.get_bibtex_from_google_scholar import GetBibTexFromGoogleScholar
if __name__ == '__main__':
google_scholar_api_key = "your_google_scholar_api_key"
get_bibtex_from_crossref = GetBibTex("1536727925@qq.com")
get_bibtex_from_google_scholar = GetBibTexFromGoogleScholar(google_scholar_api_key, GetBibTexFromGoogleScholar.APA)
with open("inputfile/Bibliographyraw.txt", "r", encoding='utf-8') as f:
raws = f.readlines()
# get bibtex from CrossRef and failed search results
success_bibtexs_crossref, failed_results = get_bibtex_from_crossref.get_bibtexs(raws)
# for each failed search result, get bibtex from Google Scholar
success_bibtexs_google, failed_results = get_bibtex_from_google_scholar.get_bibtexs(failed_results)
with open("outputfile/BibliographyCrossRef.txt", "w", encoding='utf-8') as f:
for bibtex in success_bibtexs_crossref:
f.write(bibtex)
with open("outputfile/BibliographyGoogleScholar.txt", "w", encoding='utf-8') as f:
for index, bibtex in enumerate(success_bibtexs_google):
f.write("[]".format(index) + " " + bibtex + "\n")
with open("outputfile/not_find.txt", "w", encoding='utf-8') as f:
for result in failed_results:
f.write(result+"\n")
print("find bibtex from CrossRef: ", len(success_bibtexs_crossref))
print("find bibtex from Google Scholar: ", len(success_bibtexs_google))
print("not find: ", len(failed_results))
Key code explained
Bibliographyraw.txt里面是需要查询的文件
例如：
J. Hu, L. Shen, S. Albanie, G. Sun, and A. Vedaldi, “Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks.” arXiv, Jan. 12, 2019. doi: 10.48550/arXiv.1810.12348.
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local Neural Networks.” arXiv, Apr. 13, 2018. doi: 10.48550/arXiv.1711.07971.
------------------
success_bibtexs_crossref, failed_results = get_bibtex_from_crossref.get_bibtexs(raws)
返回的第一个参数为bibtex列表，第二个为没有查询到的原文献
success_bibtexs_google, failed_results = get_bibtex_from_google_scholar.get_bibtexs(failed_results)
将没有查到的文献继续使用google API查询，一般都是可以查询到了，没有文献在google scholar查询不到了吧
提示，这里返回的其实是APA格式的，在上面初始化指定，也有一个参数可以设置返回bibtex格式，例如
get_bibtex_from_google_scholar = GetBibTexFromGoogleScholar(google_scholar_api_key, GetBibTexFromGoogleScholar.APA, flag = True)
但是需要设置代理服务器，例如：
import os
import re
import requests
os.environ["http_proxy"]="127.0.0.1:7890"
os.environ["https_proxy"]="127.0.0.1:7890"
!!!!!! Important: You need to apply for an API key first. There are 100 free queries per month, which should usually be enough. Apply at serpapi.com.
The rest of the code is self-explanatory—lol!
Of course, you can also run a single query:
get_bibtex() 去掉s就可以了
2024.4.16 Update
- Added the DBLP API
from apiModels.get_bibtex_from_dblp import GetBibTexFromDBLP
- Improved ease of use
现在提供一个封装好的类提供使用，这个方法已经封装好了Crosref和DBLP的API
from apiModels.workflow.crossref2dblp import Crossref2Dblp
使用方法（无google scholar API）
crossref2dblp = Crossref2Dblp("your email", "inputfile/Bibliographyraw.txt", "outputfile/Bibliography.txt")
crossref2dblp.running()
坐等运行完成
（有了google scholar API）
from apiModels.workflow.crossref2dblp import Crossref2Dblp
from apiModels.get_bibtex_from_google_scholar import GetBibTexFromGoogleScholar
get_bibtex_from_google_scholar = GetBibTexFromGoogleScholar(api_key="your api key")
在最后面参数加上你封装的API
crossref2dblp = Crossref2Dblp("1536727925@qq.com", "inputfile/Bibliographyraw.txt", "outputfile/Bibliography.txt",get_bibtex_from_google_scholar)
crossref2dblp.running()
坐等运行完成
或者你想自己定义api之间的调用顺序
from apiModels.workflow.make_workflow import MakeWorkflow
from apiModels.get_bibtex_from_google_scholar import GetBibTexFromGoogleScholar
from apiModels.get_bibtex_from_crossref import GetBibTex
get_bibtex_from_google_scholar = GetBibTexFromGoogleScholar(api_key="your api key")
get_bibtex_from_crossref = GetBibTex("1536727925@qq.com")
make_workflow = MakeWorkflow("inputfile/Bibliographyraw.txt", "outputfile/Bibliography.txt", get_bibtex_from_google_scholar, get_bibtex_from_crossref)
make_workflow.running()
使用之前:
pip install get_bibtex = 1.1.0
Contributions and improvements are welcome!