使用python爬取全国各地天气数据

思路

1、使用json框架解析文件_city.json[城市编码的文件]
2、使用MySQLdb连接mysql数据库
3、每个循环里面代码逻辑如下：
1）得到城市的编码
2）如果编码为空，说明是省份，使用continue进入下个循环
3）发送查看该城市天气的请求
4）使用json框架解析天气接口返回的数据
5）将该城市的天气数据导入数据库

设计脚本

代码如下：

# -*- coding: utf-8 -*-


import requests
import json
import MySQLdb
import _mysql_exceptions


def f():
    doc = open("_city.json", 'r')
    cities = json.load(doc)
    conn = MySQLdb.connect("192.168.2.105", "root", "testin123", "weather", charset='utf8')
    cursor = conn.cursor()
    for i in range(0, len(cities)):
        city = cities[i]
        if city['city_code'] == "":
            continue
        url = 
        print(url)
        response = requests.get(url)
        result = response.content
        # print(result)    # 将响应结果打印出来
        weatherdict = json.loads(result)    # 将响应结果转成字典
        try:
            cityinfo = weatherdict['cityInfo']
        except KeyError:
            print('城市编号异常')
            continue
        datadict = weatherdict['data']
        zhiliang = ""
        pm25 = 0
        pm10 = 0
        shidu = ""
        # print(datadict)    # 将最近几天的天气数据打印出来
        print(datadict['wendu'])    # 打印当前的气温
        try:
            zhiliang = datadict['quality']
        except KeyError:
            print("出现异常")
        try:
            pm25 = datadict['pm25']
        except KeyError:
            print("出现异常")
        try:
            pm10 = datadict['pm10']
        except KeyError:
            print("出现异常")
        try:
            shidu = datadict['shidu']
        except KeyError:
            print("出现异常")
        sql = 'insert into cityinfo values(%s, %s, %s, %s, %s, %s, %s, %s, %s)'
        print(sql)
        try:
            cursor.execute(sql, (cityinfo['cityId'], cityinfo['city'], cityinfo['parent'], shidu,
                                 pm25, pm10, zhiliang, datadict['wendu'],
                                 datadict['ganmao']))
            
        except _mysql_exceptions.IntegrityError:
            print("导入数据库失败")
            conn.rollback()
    conn.close()


if __name__ == "__main__":
    f()

到数据库里面查看数据

由于频繁请求天气接口，导致请求被拦截

参考资料