docker network 与 本地 network 网段冲突

现象： # 使用docker compose 挂载 named volume 无效（且没有错误提示) 排查过程： # 一开始是没有使用docker-compose命令，直接使用docker run -v 命令，挂载两个绝对路径，没有问题。

在 medium 上看到一篇很赞的文章……无奈关键部分一律无法加载出来，挂了梯子也不行，很心塞。刚刚突然发现加载出来了，以防之后再次无法访问，所以搬运过来。 There are couple of articles on how to integrate Scrapy into a Django Application (or vice versa?). But most of them don’t cover a full complete example that includes triggering spiders from Django views. Since this is a web application, that must be our main goal.

lua 是一门轻量级的脚本语言，好像比较适合写游戏？我在 太阳神三国杀 中见过很多 lua 脚本。由于 splash 的渲染脚本需要用 lua 来写，因此来学习一波。 直接上语法，看到了 python 和 pascal 的影子 orz。 详细代码 1-- Two dashes start a one-line comment. 2 3--[[ 4 Adding two ['s and ]'s makes it a 5 multi-line comment. 6--]] 7 8---------------------------------------------------- 9-- 1. Variables and flow control. 10---------------------------------------------------- 11 12num = 42 -- All numbers are doubles. 13-- Don't freak out, 64-bit doubles have 52 bits for 14-- storing exact int values; machine precision is 15-- not a problem for ints that need < 52 bits. 16 17s = 'walternate' -- Immutable strings like Python. 18t = "double-quotes are also fine" 19u = [[ Double brackets 20 start and end 21 multi-line strings.]] 22t = nil -- Undefines t; Lua has garbage collection. 23 24-- Blocks are denoted with keywords like do/end: 25while num < 50 do 26 num = num + 1 -- No ++ or += type operators. 27end 28 29-- If clauses: 30if num > 40 then 31 print('over 40') 32elseif s ~= 'walternate' then -- ~= is not equals. 33 -- Equality check is == like Python; ok for strs. 34 io.write('not over 40\n') -- Defaults to stdout. 35else 36 -- Variables are global by default. 37 thisIsGlobal = 5 -- Camel case is common. 38 39 -- How to make a variable local: 40 local line = io.read() -- Reads next stdin line. 41 42 -- String concatenation uses the .. operator: 43 print('Winter is coming, ' .. line) 44end 45 46-- Undefined variables return nil. 47-- This is not an error: 48foo = anUnknownVariable -- Now foo = nil. 49 50aBoolValue = false 51 52-- Only nil and false are falsy; 0 and '' are true! 53if not aBoolValue then print('twas false') end 54 55-- 'or' and 'and' are short-circuited. 56-- This is similar to the a?b:c operator in C/js: 57ans = aBoolValue and 'yes' or 'no' --> 'no' 58 59karlSum = 0 60for i = 1, 100 do -- The range includes both ends. 61 karlSum = karlSum + i 62end 63 64-- Use "100, 1, -1" as the range to count down: 65fredSum = 0 66for j = 100, 1, -1 do fredSum = fredSum + j end 67 68-- In general, the range is begin, end[, step]. 69 70-- Another loop construct: 71repeat 72 print('the way of the future') 73 num = num - 1 74until num == 0 75 76 77---------------------------------------------------- 78-- 2. Functions. 79---------------------------------------------------- 80 81function fib(n) 82 if n < 2 then return 1 end 83 return fib(n - 2) + fib(n - 1) 84end 85 86-- Closures and anonymous functions are ok: 87function adder(x) 88 -- The returned function is created when adder is 89 -- called, and remembers the value of x: 90 return function (y) return x + y end 91end 92a1 = adder(9) 93a2 = adder(36) 94print(a1(16)) --> 25 95print(a2(64)) --> 100 96 97-- Returns, func calls, and assignments all work 98-- with lists that may be mismatched in length. 99-- Unmatched receivers are nil; 100-- unmatched senders are discarded. 101 102x, y, z = 1, 2, 3, 4 103-- Now x = 1, y = 2, z = 3, and 4 is thrown away. 104 105function bar(a, b, c) 106 print(a, b, c) 107 return 4, 8, 15, 16, 23, 42 108end 109 110x, y = bar('zaphod') --> prints "zaphod nil nil" 111-- Now x = 4, y = 8, values 15..42 are discarded. 112 113-- Functions are first-class, may be local/global. 114-- These are the same: 115function f(x) return x * x end 116f = function (x) return x * x end 117 118-- And so are these: 119local function g(x) return math.sin(x) end 120local g; g = function (x) return math.sin(x) end 121-- the 'local g' decl makes g-self-references ok. 122 123-- Trig funcs work in radians, by the way. 124 125-- Calls with one string param don't need parens: 126print 'hello' -- Works fine. 127 128 129---------------------------------------------------- 130-- 3. Tables. 131---------------------------------------------------- 132 133-- Tables = Lua's only compound data structure; 134-- they are associative arrays. 135-- Similar to php arrays or js objects, they are 136-- hash-lookup dicts that can also be used as lists. 137 138-- Using tables as dictionaries / maps: 139 140-- Dict literals have string keys by default: 141t = {key1 = 'value1', key2 = false} 142 143-- String keys can use js-like dot notation: 144print(t.key1) -- Prints 'value1'. 145t.newKey = {} -- Adds a new key/value pair. 146t.key2 = nil -- Removes key2 from the table. 147 148-- Literal notation for any (non-nil) value as key: 149u = {['@!#'] = 'qbert', [{}] = 1729, [6.28] = 'tau'} 150print(u[6.28]) -- prints "tau" 151 152-- Key matching is basically by value for numbers 153-- and strings, but by identity for tables. 154a = u['@!#'] -- Now a = 'qbert'. 155b = u[{}] -- We might expect 1729, but it's nil: 156-- b = nil since the lookup fails. It fails 157-- because the key we used is not the same object 158-- as the one used to store the original value. So 159-- strings & numbers are more portable keys. 160 161-- A one-table-param function call needs no parens: 162function h(x) print(x.key1) end 163h{key1 = 'Sonmi~451'} -- Prints 'Sonmi~451'. 164 165for key, val in pairs(u) do -- Table iteration. 166 print(key, val) 167end 168 169-- _G is a special table of all globals. 170print(_G['_G'] == _G) -- Prints 'true'. 171 172-- Using tables as lists / arrays: 173 174-- List literals implicitly set up int keys: 175v = {'value1', 'value2', 1.21, 'gigawatts'} 176for i = 1, #v do -- #v is the size of v for lists. 177 print(v[i]) -- Indices start at 1 !! SO CRAZY! 178end 179-- A 'list' is not a real type. v is just a table 180-- with consecutive integer keys, treated as a list. 181 182---------------------------------------------------- 183-- 3.1 Metatables and metamethods. 184---------------------------------------------------- 185 186-- A table can have a metatable that gives the table 187-- operator-overloadish behavior. Later we'll see 188-- how metatables support js-prototypey behavior. 189 190f1 = {a = 1, b = 2} -- Represents the fraction a/b. 191f2 = {a = 2, b = 3} 192 193-- This would fail: 194-- s = f1 + f2 195 196metafraction = {} 197function metafraction.__add(f1, f2) 198 sum = {} 199 sum.b = f1.b * f2.b 200 sum.a = f1.a * f2.b + f2.a * f1.b 201 return sum 202end 203 204setmetatable(f1, metafraction) 205setmetatable(f2, metafraction) 206 207s = f1 + f2 -- call __add(f1, f2) on f1's metatable 208 209-- f1, f2 have no key for their metatable, unlike 210-- prototypes in js, so you must retrieve it as in 211-- getmetatable(f1). The metatable is a normal table 212-- with keys that Lua knows about, like __add. 213 214-- But the next line fails since s has no metatable: 215-- t = s + s 216-- Class-like patterns given below would fix this. 217 218-- An __index on a metatable overloads dot lookups: 219defaultFavs = {animal = 'gru', food = 'donuts'} 220myFavs = {food = 'pizza'} 221setmetatable(myFavs, {__index = defaultFavs}) 222eatenBy = myFavs.animal -- works! thanks, metatable 223 224-- Direct table lookups that fail will retry using 225-- the metatable's __index value, and this recurses. 226 227-- An __index value can also be a function(tbl, key) 228-- for more customized lookups. 229 230-- Values of __index,add, .. are called metamethods. 231-- Full list. Here a is a table with the metamethod. 232 233-- __add(a, b) for a + b 234-- __sub(a, b) for a - b 235-- __mul(a, b) for a * b 236-- __div(a, b) for a / b 237-- __mod(a, b) for a % b 238-- __pow(a, b) for a ^ b 239-- __unm(a) for -a 240-- __concat(a, b) for a .. b 241-- __len(a) for #a 242-- __eq(a, b) for a == b 243-- __lt(a, b) for a < b 244-- __le(a, b) for a <= b 245-- __index(a, b) <fn or a table> for a.b 246-- __newindex(a, b, c) for a.b = c 247-- __call(a, ...) for a(...) 248 249---------------------------------------------------- 250-- 3.2 Class-like tables and inheritance. 251---------------------------------------------------- 252 253-- Classes aren't built in; there are different ways 254-- to make them using tables and metatables. 255 256-- Explanation for this example is below it. 257 258Dog = {} -- 1. 259 260function Dog:new() -- 2. 261 newObj = {sound = 'woof'} -- 3. 262 self.__index = self -- 4. 263 return setmetatable(newObj, self) -- 5. 264end 265 266function Dog:makeSound() -- 6. 267 print('I say ' .. self.sound) 268end 269 270mrDog = Dog:new() -- 7. 271mrDog:makeSound() -- 'I say woof' -- 8. 272 273-- 1. Dog acts like a class; it's really a table. 274-- 2. function tablename:fn(...) is the same as 275-- function tablename.fn(self, ...) 276-- The : just adds a first arg called self. 277-- Read 7 & 8 below for how self gets its value. 278-- 3. newObj will be an instance of class Dog. 279-- 4. self = the class being instantiated. Often 280-- self = Dog, but inheritance can change it. 281-- newObj gets self's functions when we set both 282-- newObj's metatable and self's __index to self. 283-- 5. Reminder: setmetatable returns its first arg. 284-- 6. The : works as in 2, but this time we expect 285-- self to be an instance instead of a class. 286-- 7. Same as Dog.new(Dog), so self = Dog in new(). 287-- 8. Same as mrDog.makeSound(mrDog); self = mrDog. 288 289---------------------------------------------------- 290 291-- Inheritance example: 292 293LoudDog = Dog:new() -- 1. 294 295function LoudDog:makeSound() 296 s = self.sound .. ' ' -- 2. 297 print(s .. s .. s) 298end 299 300seymour = LoudDog:new() -- 3. 301seymour:makeSound() -- 'woof woof woof' -- 4. 302 303-- 1. LoudDog gets Dog's methods and variables. 304-- 2. self has a 'sound' key from new(), see 3. 305-- 3. Same as LoudDog.new(LoudDog), and converted to 306-- Dog.new(LoudDog) as LoudDog has no 'new' key, 307-- but does have __index = Dog on its metatable. 308-- Result: seymour's metatable is LoudDog, and 309-- LoudDog.__index = LoudDog. So seymour.key will 310-- = seymour.key, LoudDog.key, Dog.key, whichever 311-- table is the first with the given key. 312-- 4. The 'makeSound' key is found in LoudDog; this 313-- is the same as LoudDog.makeSound(seymour). 314 315-- If needed, a subclass's new() is like the base's: 316function LoudDog:new() 317 newObj = {} 318 -- set up newObj 319 self.__index = self 320 return setmetatable(newObj, self) 321end 322 323---------------------------------------------------- 324-- 4. Modules. 325---------------------------------------------------- 326 327 328--[[ I'm commenting out this section so the rest of 329-- this script remains runnable. 330-- Suppose the file mod.lua looks like this: 331local M = {} 332 333local function sayMyName() 334 print('Hrunkner') 335end 336 337function M.sayHello() 338 print('Why hello there') 339 sayMyName() 340end 341 342return M 343 344-- Another file can use mod.lua's functionality: 345local mod = require('mod') -- Run the file mod.lua. 346 347-- require is the standard way to include modules. 348-- require acts like: (if not cached; see below) 349local mod = (function () 350 <contents of mod.lua> 351end)() 352-- It's like mod.lua is a function body, so that 353-- locals inside mod.lua are invisible outside it. 354 355-- This works because mod here = M in mod.lua: 356mod.sayHello() -- Says hello to Hrunkner. 357 358-- This is wrong; sayMyName only exists in mod.lua: 359mod.sayMyName() -- error 360 361-- require's return values are cached so a file is 362-- run at most once, even when require'd many times. 363 364-- Suppose mod2.lua contains "print('Hi!')". 365local a = require('mod2') -- Prints Hi! 366local b = require('mod2') -- Doesn't print; a=b. 367 368-- dofile is like require without caching: 369dofile('mod2.lua') --> Hi! 370dofile('mod2.lua') --> Hi! (runs it again) 371 372-- loadfile loads a lua file but doesn't run it yet. 373f = loadfile('mod2.lua') -- Call f() to run it. 374 375-- loadstring is loadfile for strings. 376g = loadstring('print(343)') -- Returns a function. 377g() -- Prints out 343; nothing printed before now. 378 379--]] 380 381---------------------------------------------------- 382-- 5. References. 383---------------------------------------------------- 384 385--[[ 386 387I was excited to learn Lua so I could make games 388with the Löve 2D game engine. That's the why. 389 390I started with BlackBulletIV's Lua for programmers. 391Next I read the official Programming in Lua book. 392That's the how. 393 394It might be helpful to check out the Lua short 395reference on lua-users.org. 396 397The main topics not covered are standard libraries: 398 * string library 399 * table library 400 * math library 401 * io library 402 * os library 403 404By the way, this entire file is valid Lua; save it 405as learn.lua and run it with "lua learn.lua" ! 406 407This was first written for tylerneylon.com. It's 408also available as a github gist. Tutorials for other 409languages, in the same style as this one, are here: 410 411http://learnxinyminutes.com/ 412 413Have fun with Lua! 414 415--]]

先放资料，可能比较侧重于go在系统调用方面的内容. 这里不会记录详细的go的语法，只会记录学习的过程，踩到的坑，以及其他我认为值得记录的内容. go的switch语句终于是人类思维的语句了…匹配中了不需要加break..

再次迫于生计。。。 参考了面向新人的 Python 爬虫学习资料 大致的学习路线为： 一： 简单的定向脚本爬虫（ request — bs4 — re ） 二： 大型框架式爬虫（ Scrapy 框架为主） 三：浏览器模拟爬虫 （ Mechanize 模拟 和 Selenium 模拟） 有 Python 基础和一点 html 基础的话，貌似上手是 0 难度的。