psycopg2.errors.InFailedSqlTransaction:当前事务被中止,命令被忽略,直到事务块结束
我写了一个scrapy程序来从一个站点抓取数据,如果我抓取到一个json文件或csv文件,这个程序会成功抓取,当我尝试抓取我的postgres数据库时出现问题,下图显示了我收到的错误, 如何修复错误:
def process_item(self, item, spider):
"""Save deals in the database.
This method is called for every item pipeline component.
"""
self.cur.execute("insert into Deals (Name,Deal_Url,Image_Url,Old_Price,Special_Price,Final_Price) values(%s,%s,%s,%s,%s,%s)",(item['Name'],item['Product_URL'],item['Image_URL'],item['Old_Price'],item['Special_Price'],item['Final_Price']))
self.connection.commit()
return item
回答
当事务中的操作失败时,您需要回滚,除了修复恶意操作(可能试图访问item不存在的任何值)之外,您还可以将插入语句包装在 try/except 块中:
def process_item(self, item, spider):
"""Save deals in the database.
This method is called for every item pipeline component.
"""
try:
self.cur.execute("insert into Deals (Name,Deal_Url,Image_Url,Old_Price,Special_Price,Final_Price) values(%s,%s,%s,%s,%s,%s)",(item['Name'],item['Product_URL'],item['Image_URL'],item['Old_Price'],item['Special_Price'],item['Final_Price']))
self.connection.commit()
except:
self.connection.rollback()
return item
THE END
二维码