PageRenderTime 51ms CodeModel.GetById 15ms RepoModel.GetById 0ms app.codeStats 0ms

/tags/release-0.2.0-rc0/hive/external/ql/src/test/results/clientpositive/auto_join30.q.out

#
text | 2802 lines | 2701 code | 101 blank | 0 comment | 0 complexity | 0d1b451dae175af7c0a6173d40c5f9a2 MD5 | raw file
Possible License(s): Apache-2.0, BSD-3-Clause, JSON, CPL-1.0

Large files files are truncated, but you can click here to view the full file

  1. PREHOOK: query: explain
  2. FROM
  3. (SELECT src.* FROM src sort by key) x
  4. JOIN
  5. (SELECT src.* FROM src sort by value) Y
  6. ON (x.key = Y.key)
  7. select sum(hash(Y.key,Y.value))
  8. PREHOOK: type: QUERY
  9. POSTHOOK: query: explain
  10. FROM
  11. (SELECT src.* FROM src sort by key) x
  12. JOIN
  13. (SELECT src.* FROM src sort by value) Y
  14. ON (x.key = Y.key)
  15. select sum(hash(Y.key,Y.value))
  16. POSTHOOK: type: QUERY
  17. ABSTRACT SYNTAX TREE:
  18. (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key))))) x) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Y) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL Y) key) (. (TOK_TABLE_OR_COL Y) value)))))))
  19. STAGE DEPENDENCIES:
  20. Stage-1 is a root stage
  21. Stage-7 depends on stages: Stage-1, Stage-4 , consists of Stage-8, Stage-9, Stage-2
  22. Stage-8 has a backup stage: Stage-2
  23. Stage-5 depends on stages: Stage-8
  24. Stage-3 depends on stages: Stage-2, Stage-5, Stage-6
  25. Stage-9 has a backup stage: Stage-2
  26. Stage-6 depends on stages: Stage-9
  27. Stage-2
  28. Stage-4 is a root stage
  29. Stage-0 is a root stage
  30. STAGE PLANS:
  31. Stage: Stage-1
  32. Map Reduce
  33. Alias -> Map Operator Tree:
  34. x:src
  35. TableScan
  36. alias: src
  37. Select Operator
  38. expressions:
  39. expr: key
  40. type: string
  41. expr: value
  42. type: string
  43. outputColumnNames: _col0, _col1
  44. Reduce Output Operator
  45. key expressions:
  46. expr: _col0
  47. type: string
  48. sort order: +
  49. tag: -1
  50. value expressions:
  51. expr: _col0
  52. type: string
  53. expr: _col1
  54. type: string
  55. Reduce Operator Tree:
  56. Extract
  57. File Output Operator
  58. compressed: false
  59. GlobalTableId: 0
  60. table:
  61. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  62. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  63. Stage: Stage-7
  64. Conditional Operator
  65. Stage: Stage-8
  66. Map Reduce Local Work
  67. Alias -> Map Local Tables:
  68. $INTNAME1
  69. Fetch Operator
  70. limit: -1
  71. Alias -> Map Local Operator Tree:
  72. $INTNAME1
  73. HashTable Sink Operator
  74. condition expressions:
  75. 0
  76. 1 {_col0} {_col1}
  77. handleSkewJoin: false
  78. keys:
  79. 0 [Column[_col0]]
  80. 1 [Column[_col0]]
  81. Position of Big Table: 0
  82. Stage: Stage-5
  83. Map Reduce
  84. Alias -> Map Operator Tree:
  85. $INTNAME
  86. Map Join Operator
  87. condition map:
  88. Inner Join 0 to 1
  89. condition expressions:
  90. 0
  91. 1 {_col0} {_col1}
  92. handleSkewJoin: false
  93. keys:
  94. 0 [Column[_col0]]
  95. 1 [Column[_col0]]
  96. outputColumnNames: _col2, _col3
  97. Position of Big Table: 0
  98. Select Operator
  99. expressions:
  100. expr: _col2
  101. type: string
  102. expr: _col3
  103. type: string
  104. outputColumnNames: _col2, _col3
  105. Group By Operator
  106. aggregations:
  107. expr: sum(hash(_col2,_col3))
  108. bucketGroup: false
  109. mode: hash
  110. outputColumnNames: _col0
  111. File Output Operator
  112. compressed: false
  113. GlobalTableId: 0
  114. table:
  115. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  116. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  117. Local Work:
  118. Map Reduce Local Work
  119. Stage: Stage-3
  120. Map Reduce
  121. Alias -> Map Operator Tree:
  122. file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-29-33_521_1049482282230510027/-mr-10003
  123. Reduce Output Operator
  124. sort order:
  125. tag: -1
  126. value expressions:
  127. expr: _col0
  128. type: bigint
  129. Reduce Operator Tree:
  130. Group By Operator
  131. aggregations:
  132. expr: sum(VALUE._col0)
  133. bucketGroup: false
  134. mode: mergepartial
  135. outputColumnNames: _col0
  136. Select Operator
  137. expressions:
  138. expr: _col0
  139. type: bigint
  140. outputColumnNames: _col0
  141. File Output Operator
  142. compressed: false
  143. GlobalTableId: 0
  144. table:
  145. input format: org.apache.hadoop.mapred.TextInputFormat
  146. output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
  147. Stage: Stage-9
  148. Map Reduce Local Work
  149. Alias -> Map Local Tables:
  150. $INTNAME
  151. Fetch Operator
  152. limit: -1
  153. Alias -> Map Local Operator Tree:
  154. $INTNAME
  155. HashTable Sink Operator
  156. condition expressions:
  157. 0
  158. 1 {_col0} {_col1}
  159. handleSkewJoin: false
  160. keys:
  161. 0 [Column[_col0]]
  162. 1 [Column[_col0]]
  163. Position of Big Table: 1
  164. Stage: Stage-6
  165. Map Reduce
  166. Alias -> Map Operator Tree:
  167. $INTNAME1
  168. Map Join Operator
  169. condition map:
  170. Inner Join 0 to 1
  171. condition expressions:
  172. 0
  173. 1 {_col0} {_col1}
  174. handleSkewJoin: false
  175. keys:
  176. 0 [Column[_col0]]
  177. 1 [Column[_col0]]
  178. outputColumnNames: _col2, _col3
  179. Position of Big Table: 1
  180. Select Operator
  181. expressions:
  182. expr: _col2
  183. type: string
  184. expr: _col3
  185. type: string
  186. outputColumnNames: _col2, _col3
  187. Group By Operator
  188. aggregations:
  189. expr: sum(hash(_col2,_col3))
  190. bucketGroup: false
  191. mode: hash
  192. outputColumnNames: _col0
  193. File Output Operator
  194. compressed: false
  195. GlobalTableId: 0
  196. table:
  197. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  198. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  199. Local Work:
  200. Map Reduce Local Work
  201. Stage: Stage-2
  202. Map Reduce
  203. Alias -> Map Operator Tree:
  204. $INTNAME
  205. Reduce Output Operator
  206. key expressions:
  207. expr: _col0
  208. type: string
  209. sort order: +
  210. Map-reduce partition columns:
  211. expr: _col0
  212. type: string
  213. tag: 0
  214. $INTNAME1
  215. Reduce Output Operator
  216. key expressions:
  217. expr: _col0
  218. type: string
  219. sort order: +
  220. Map-reduce partition columns:
  221. expr: _col0
  222. type: string
  223. tag: 1
  224. value expressions:
  225. expr: _col0
  226. type: string
  227. expr: _col1
  228. type: string
  229. Reduce Operator Tree:
  230. Join Operator
  231. condition map:
  232. Inner Join 0 to 1
  233. condition expressions:
  234. 0
  235. 1 {VALUE._col0} {VALUE._col1}
  236. handleSkewJoin: false
  237. outputColumnNames: _col2, _col3
  238. Select Operator
  239. expressions:
  240. expr: _col2
  241. type: string
  242. expr: _col3
  243. type: string
  244. outputColumnNames: _col2, _col3
  245. Group By Operator
  246. aggregations:
  247. expr: sum(hash(_col2,_col3))
  248. bucketGroup: false
  249. mode: hash
  250. outputColumnNames: _col0
  251. File Output Operator
  252. compressed: false
  253. GlobalTableId: 0
  254. table:
  255. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  256. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  257. Stage: Stage-4
  258. Map Reduce
  259. Alias -> Map Operator Tree:
  260. y:src
  261. TableScan
  262. alias: src
  263. Select Operator
  264. expressions:
  265. expr: key
  266. type: string
  267. expr: value
  268. type: string
  269. outputColumnNames: _col0, _col1
  270. Reduce Output Operator
  271. key expressions:
  272. expr: _col1
  273. type: string
  274. sort order: +
  275. tag: -1
  276. value expressions:
  277. expr: _col0
  278. type: string
  279. expr: _col1
  280. type: string
  281. Reduce Operator Tree:
  282. Extract
  283. File Output Operator
  284. compressed: false
  285. GlobalTableId: 0
  286. table:
  287. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  288. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  289. Stage: Stage-0
  290. Fetch Operator
  291. limit: -1
  292. PREHOOK: query: FROM
  293. (SELECT src.* FROM src sort by key) x
  294. JOIN
  295. (SELECT src.* FROM src sort by value) Y
  296. ON (x.key = Y.key)
  297. select sum(hash(Y.key,Y.value))
  298. PREHOOK: type: QUERY
  299. PREHOOK: Input: default@src
  300. PREHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-29-35_206_4517835727241747847/-mr-10000
  301. POSTHOOK: query: FROM
  302. (SELECT src.* FROM src sort by key) x
  303. JOIN
  304. (SELECT src.* FROM src sort by value) Y
  305. ON (x.key = Y.key)
  306. select sum(hash(Y.key,Y.value))
  307. POSTHOOK: type: QUERY
  308. POSTHOOK: Input: default@src
  309. POSTHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-29-35_206_4517835727241747847/-mr-10000
  310. 103231310608
  311. PREHOOK: query: explain
  312. FROM
  313. (SELECT src.* FROM src sort by key) x
  314. LEFT OUTER JOIN
  315. (SELECT src.* FROM src sort by value) Y
  316. ON (x.key = Y.key)
  317. select sum(hash(Y.key,Y.value))
  318. PREHOOK: type: QUERY
  319. POSTHOOK: query: explain
  320. FROM
  321. (SELECT src.* FROM src sort by key) x
  322. LEFT OUTER JOIN
  323. (SELECT src.* FROM src sort by value) Y
  324. ON (x.key = Y.key)
  325. select sum(hash(Y.key,Y.value))
  326. POSTHOOK: type: QUERY
  327. ABSTRACT SYNTAX TREE:
  328. (TOK_QUERY (TOK_FROM (TOK_LEFTOUTERJOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key))))) x) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Y) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL Y) key) (. (TOK_TABLE_OR_COL Y) value)))))))
  329. STAGE DEPENDENCIES:
  330. Stage-1 is a root stage
  331. Stage-6 depends on stages: Stage-1, Stage-4 , consists of Stage-7, Stage-2
  332. Stage-7 has a backup stage: Stage-2
  333. Stage-5 depends on stages: Stage-7
  334. Stage-3 depends on stages: Stage-2, Stage-5
  335. Stage-2
  336. Stage-4 is a root stage
  337. Stage-0 is a root stage
  338. STAGE PLANS:
  339. Stage: Stage-1
  340. Map Reduce
  341. Alias -> Map Operator Tree:
  342. x:src
  343. TableScan
  344. alias: src
  345. Select Operator
  346. expressions:
  347. expr: key
  348. type: string
  349. expr: value
  350. type: string
  351. outputColumnNames: _col0, _col1
  352. Reduce Output Operator
  353. key expressions:
  354. expr: _col0
  355. type: string
  356. sort order: +
  357. tag: -1
  358. value expressions:
  359. expr: _col0
  360. type: string
  361. expr: _col1
  362. type: string
  363. Reduce Operator Tree:
  364. Extract
  365. File Output Operator
  366. compressed: false
  367. GlobalTableId: 0
  368. table:
  369. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  370. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  371. Stage: Stage-6
  372. Conditional Operator
  373. Stage: Stage-7
  374. Map Reduce Local Work
  375. Alias -> Map Local Tables:
  376. $INTNAME1
  377. Fetch Operator
  378. limit: -1
  379. Alias -> Map Local Operator Tree:
  380. $INTNAME1
  381. HashTable Sink Operator
  382. condition expressions:
  383. 0
  384. 1 {_col0} {_col1}
  385. handleSkewJoin: false
  386. keys:
  387. 0 [Column[_col0]]
  388. 1 [Column[_col0]]
  389. Position of Big Table: 0
  390. Stage: Stage-5
  391. Map Reduce
  392. Alias -> Map Operator Tree:
  393. $INTNAME
  394. Map Join Operator
  395. condition map:
  396. Left Outer Join0 to 1
  397. condition expressions:
  398. 0
  399. 1 {_col0} {_col1}
  400. handleSkewJoin: false
  401. keys:
  402. 0 [Column[_col0]]
  403. 1 [Column[_col0]]
  404. outputColumnNames: _col2, _col3
  405. Position of Big Table: 0
  406. Select Operator
  407. expressions:
  408. expr: _col2
  409. type: string
  410. expr: _col3
  411. type: string
  412. outputColumnNames: _col2, _col3
  413. Group By Operator
  414. aggregations:
  415. expr: sum(hash(_col2,_col3))
  416. bucketGroup: false
  417. mode: hash
  418. outputColumnNames: _col0
  419. File Output Operator
  420. compressed: false
  421. GlobalTableId: 0
  422. table:
  423. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  424. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  425. Local Work:
  426. Map Reduce Local Work
  427. Stage: Stage-3
  428. Map Reduce
  429. Alias -> Map Operator Tree:
  430. file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-30-12_697_8776019509777494227/-mr-10003
  431. Reduce Output Operator
  432. sort order:
  433. tag: -1
  434. value expressions:
  435. expr: _col0
  436. type: bigint
  437. Reduce Operator Tree:
  438. Group By Operator
  439. aggregations:
  440. expr: sum(VALUE._col0)
  441. bucketGroup: false
  442. mode: mergepartial
  443. outputColumnNames: _col0
  444. Select Operator
  445. expressions:
  446. expr: _col0
  447. type: bigint
  448. outputColumnNames: _col0
  449. File Output Operator
  450. compressed: false
  451. GlobalTableId: 0
  452. table:
  453. input format: org.apache.hadoop.mapred.TextInputFormat
  454. output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
  455. Stage: Stage-2
  456. Map Reduce
  457. Alias -> Map Operator Tree:
  458. $INTNAME
  459. Reduce Output Operator
  460. key expressions:
  461. expr: _col0
  462. type: string
  463. sort order: +
  464. Map-reduce partition columns:
  465. expr: _col0
  466. type: string
  467. tag: 0
  468. $INTNAME1
  469. Reduce Output Operator
  470. key expressions:
  471. expr: _col0
  472. type: string
  473. sort order: +
  474. Map-reduce partition columns:
  475. expr: _col0
  476. type: string
  477. tag: 1
  478. value expressions:
  479. expr: _col0
  480. type: string
  481. expr: _col1
  482. type: string
  483. Reduce Operator Tree:
  484. Join Operator
  485. condition map:
  486. Left Outer Join0 to 1
  487. condition expressions:
  488. 0
  489. 1 {VALUE._col0} {VALUE._col1}
  490. handleSkewJoin: false
  491. outputColumnNames: _col2, _col3
  492. Select Operator
  493. expressions:
  494. expr: _col2
  495. type: string
  496. expr: _col3
  497. type: string
  498. outputColumnNames: _col2, _col3
  499. Group By Operator
  500. aggregations:
  501. expr: sum(hash(_col2,_col3))
  502. bucketGroup: false
  503. mode: hash
  504. outputColumnNames: _col0
  505. File Output Operator
  506. compressed: false
  507. GlobalTableId: 0
  508. table:
  509. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  510. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  511. Stage: Stage-4
  512. Map Reduce
  513. Alias -> Map Operator Tree:
  514. y:src
  515. TableScan
  516. alias: src
  517. Select Operator
  518. expressions:
  519. expr: key
  520. type: string
  521. expr: value
  522. type: string
  523. outputColumnNames: _col0, _col1
  524. Reduce Output Operator
  525. key expressions:
  526. expr: _col1
  527. type: string
  528. sort order: +
  529. tag: -1
  530. value expressions:
  531. expr: _col0
  532. type: string
  533. expr: _col1
  534. type: string
  535. Reduce Operator Tree:
  536. Extract
  537. File Output Operator
  538. compressed: false
  539. GlobalTableId: 0
  540. table:
  541. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  542. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  543. Stage: Stage-0
  544. Fetch Operator
  545. limit: -1
  546. PREHOOK: query: FROM
  547. (SELECT src.* FROM src sort by key) x
  548. LEFT OUTER JOIN
  549. (SELECT src.* FROM src sort by value) Y
  550. ON (x.key = Y.key)
  551. select sum(hash(Y.key,Y.value))
  552. PREHOOK: type: QUERY
  553. PREHOOK: Input: default@src
  554. PREHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-30-13_659_8930056926219050163/-mr-10000
  555. POSTHOOK: query: FROM
  556. (SELECT src.* FROM src sort by key) x
  557. LEFT OUTER JOIN
  558. (SELECT src.* FROM src sort by value) Y
  559. ON (x.key = Y.key)
  560. select sum(hash(Y.key,Y.value))
  561. POSTHOOK: type: QUERY
  562. POSTHOOK: Input: default@src
  563. POSTHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-30-13_659_8930056926219050163/-mr-10000
  564. 103231310608
  565. PREHOOK: query: explain
  566. FROM
  567. (SELECT src.* FROM src sort by key) x
  568. RIGHT OUTER JOIN
  569. (SELECT src.* FROM src sort by value) Y
  570. ON (x.key = Y.key)
  571. select sum(hash(Y.key,Y.value))
  572. PREHOOK: type: QUERY
  573. POSTHOOK: query: explain
  574. FROM
  575. (SELECT src.* FROM src sort by key) x
  576. RIGHT OUTER JOIN
  577. (SELECT src.* FROM src sort by value) Y
  578. ON (x.key = Y.key)
  579. select sum(hash(Y.key,Y.value))
  580. POSTHOOK: type: QUERY
  581. ABSTRACT SYNTAX TREE:
  582. (TOK_QUERY (TOK_FROM (TOK_RIGHTOUTERJOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key))))) x) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Y) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL Y) key) (. (TOK_TABLE_OR_COL Y) value)))))))
  583. STAGE DEPENDENCIES:
  584. Stage-1 is a root stage
  585. Stage-6 depends on stages: Stage-1, Stage-4 , consists of Stage-7, Stage-2
  586. Stage-7 has a backup stage: Stage-2
  587. Stage-5 depends on stages: Stage-7
  588. Stage-3 depends on stages: Stage-2, Stage-5
  589. Stage-2
  590. Stage-4 is a root stage
  591. Stage-0 is a root stage
  592. STAGE PLANS:
  593. Stage: Stage-1
  594. Map Reduce
  595. Alias -> Map Operator Tree:
  596. x:src
  597. TableScan
  598. alias: src
  599. Select Operator
  600. expressions:
  601. expr: key
  602. type: string
  603. expr: value
  604. type: string
  605. outputColumnNames: _col0, _col1
  606. Reduce Output Operator
  607. key expressions:
  608. expr: _col0
  609. type: string
  610. sort order: +
  611. tag: -1
  612. value expressions:
  613. expr: _col0
  614. type: string
  615. expr: _col1
  616. type: string
  617. Reduce Operator Tree:
  618. Extract
  619. File Output Operator
  620. compressed: false
  621. GlobalTableId: 0
  622. table:
  623. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  624. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  625. Stage: Stage-6
  626. Conditional Operator
  627. Stage: Stage-7
  628. Map Reduce Local Work
  629. Alias -> Map Local Tables:
  630. $INTNAME
  631. Fetch Operator
  632. limit: -1
  633. Alias -> Map Local Operator Tree:
  634. $INTNAME
  635. HashTable Sink Operator
  636. condition expressions:
  637. 0
  638. 1 {_col0} {_col1}
  639. handleSkewJoin: false
  640. keys:
  641. 0 [Column[_col0]]
  642. 1 [Column[_col0]]
  643. Position of Big Table: 1
  644. Stage: Stage-5
  645. Map Reduce
  646. Alias -> Map Operator Tree:
  647. $INTNAME1
  648. Map Join Operator
  649. condition map:
  650. Right Outer Join0 to 1
  651. condition expressions:
  652. 0
  653. 1 {_col0} {_col1}
  654. handleSkewJoin: false
  655. keys:
  656. 0 [Column[_col0]]
  657. 1 [Column[_col0]]
  658. outputColumnNames: _col2, _col3
  659. Position of Big Table: 1
  660. Select Operator
  661. expressions:
  662. expr: _col2
  663. type: string
  664. expr: _col3
  665. type: string
  666. outputColumnNames: _col2, _col3
  667. Group By Operator
  668. aggregations:
  669. expr: sum(hash(_col2,_col3))
  670. bucketGroup: false
  671. mode: hash
  672. outputColumnNames: _col0
  673. File Output Operator
  674. compressed: false
  675. GlobalTableId: 0
  676. table:
  677. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  678. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  679. Local Work:
  680. Map Reduce Local Work
  681. Stage: Stage-3
  682. Map Reduce
  683. Alias -> Map Operator Tree:
  684. file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-30-45_367_2576785772484562232/-mr-10003
  685. Reduce Output Operator
  686. sort order:
  687. tag: -1
  688. value expressions:
  689. expr: _col0
  690. type: bigint
  691. Reduce Operator Tree:
  692. Group By Operator
  693. aggregations:
  694. expr: sum(VALUE._col0)
  695. bucketGroup: false
  696. mode: mergepartial
  697. outputColumnNames: _col0
  698. Select Operator
  699. expressions:
  700. expr: _col0
  701. type: bigint
  702. outputColumnNames: _col0
  703. File Output Operator
  704. compressed: false
  705. GlobalTableId: 0
  706. table:
  707. input format: org.apache.hadoop.mapred.TextInputFormat
  708. output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
  709. Stage: Stage-2
  710. Map Reduce
  711. Alias -> Map Operator Tree:
  712. $INTNAME
  713. Reduce Output Operator
  714. key expressions:
  715. expr: _col0
  716. type: string
  717. sort order: +
  718. Map-reduce partition columns:
  719. expr: _col0
  720. type: string
  721. tag: 0
  722. $INTNAME1
  723. Reduce Output Operator
  724. key expressions:
  725. expr: _col0
  726. type: string
  727. sort order: +
  728. Map-reduce partition columns:
  729. expr: _col0
  730. type: string
  731. tag: 1
  732. value expressions:
  733. expr: _col0
  734. type: string
  735. expr: _col1
  736. type: string
  737. Reduce Operator Tree:
  738. Join Operator
  739. condition map:
  740. Right Outer Join0 to 1
  741. condition expressions:
  742. 0
  743. 1 {VALUE._col0} {VALUE._col1}
  744. handleSkewJoin: false
  745. outputColumnNames: _col2, _col3
  746. Select Operator
  747. expressions:
  748. expr: _col2
  749. type: string
  750. expr: _col3
  751. type: string
  752. outputColumnNames: _col2, _col3
  753. Group By Operator
  754. aggregations:
  755. expr: sum(hash(_col2,_col3))
  756. bucketGroup: false
  757. mode: hash
  758. outputColumnNames: _col0
  759. File Output Operator
  760. compressed: false
  761. GlobalTableId: 0
  762. table:
  763. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  764. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  765. Stage: Stage-4
  766. Map Reduce
  767. Alias -> Map Operator Tree:
  768. y:src
  769. TableScan
  770. alias: src
  771. Select Operator
  772. expressions:
  773. expr: key
  774. type: string
  775. expr: value
  776. type: string
  777. outputColumnNames: _col0, _col1
  778. Reduce Output Operator
  779. key expressions:
  780. expr: _col1
  781. type: string
  782. sort order: +
  783. tag: -1
  784. value expressions:
  785. expr: _col0
  786. type: string
  787. expr: _col1
  788. type: string
  789. Reduce Operator Tree:
  790. Extract
  791. File Output Operator
  792. compressed: false
  793. GlobalTableId: 0
  794. table:
  795. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  796. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  797. Stage: Stage-0
  798. Fetch Operator
  799. limit: -1
  800. PREHOOK: query: FROM
  801. (SELECT src.* FROM src sort by key) x
  802. RIGHT OUTER JOIN
  803. (SELECT src.* FROM src sort by value) Y
  804. ON (x.key = Y.key)
  805. select sum(hash(Y.key,Y.value))
  806. PREHOOK: type: QUERY
  807. PREHOOK: Input: default@src
  808. PREHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-30-45_713_4388632692386585223/-mr-10000
  809. POSTHOOK: query: FROM
  810. (SELECT src.* FROM src sort by key) x
  811. RIGHT OUTER JOIN
  812. (SELECT src.* FROM src sort by value) Y
  813. ON (x.key = Y.key)
  814. select sum(hash(Y.key,Y.value))
  815. POSTHOOK: type: QUERY
  816. POSTHOOK: Input: default@src
  817. POSTHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-30-45_713_4388632692386585223/-mr-10000
  818. 103231310608
  819. PREHOOK: query: explain
  820. FROM
  821. (SELECT src.* FROM src sort by key) x
  822. JOIN
  823. (SELECT src.* FROM src sort by value) Y
  824. ON (x.key = Y.key)
  825. JOIN
  826. (SELECT src.* FROM src sort by value) Z
  827. ON (x.key = Z.key)
  828. select sum(hash(Y.key,Y.value))
  829. PREHOOK: type: QUERY
  830. POSTHOOK: query: explain
  831. FROM
  832. (SELECT src.* FROM src sort by key) x
  833. JOIN
  834. (SELECT src.* FROM src sort by value) Y
  835. ON (x.key = Y.key)
  836. JOIN
  837. (SELECT src.* FROM src sort by value) Z
  838. ON (x.key = Z.key)
  839. select sum(hash(Y.key,Y.value))
  840. POSTHOOK: type: QUERY
  841. ABSTRACT SYNTAX TREE:
  842. (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key))))) x) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Y) key))) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Z) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Z) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL Y) key) (. (TOK_TABLE_OR_COL Y) value)))))))
  843. STAGE DEPENDENCIES:
  844. Stage-1 is a root stage
  845. Stage-9 depends on stages: Stage-1, Stage-4, Stage-5 , consists of Stage-10, Stage-11, Stage-12, Stage-2
  846. Stage-10 has a backup stage: Stage-2
  847. Stage-6 depends on stages: Stage-10
  848. Stage-3 depends on stages: Stage-2, Stage-6, Stage-7, Stage-8
  849. Stage-11 has a backup stage: Stage-2
  850. Stage-7 depends on stages: Stage-11
  851. Stage-12 has a backup stage: Stage-2
  852. Stage-8 depends on stages: Stage-12
  853. Stage-2
  854. Stage-4 is a root stage
  855. Stage-5 is a root stage
  856. Stage-0 is a root stage
  857. STAGE PLANS:
  858. Stage: Stage-1
  859. Map Reduce
  860. Alias -> Map Operator Tree:
  861. z:src
  862. TableScan
  863. alias: src
  864. Select Operator
  865. expressions:
  866. expr: key
  867. type: string
  868. expr: value
  869. type: string
  870. outputColumnNames: _col0, _col1
  871. Reduce Output Operator
  872. key expressions:
  873. expr: _col1
  874. type: string
  875. sort order: +
  876. tag: -1
  877. value expressions:
  878. expr: _col0
  879. type: string
  880. expr: _col1
  881. type: string
  882. Reduce Operator Tree:
  883. Extract
  884. File Output Operator
  885. compressed: false
  886. GlobalTableId: 0
  887. table:
  888. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  889. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  890. Stage: Stage-9
  891. Conditional Operator
  892. Stage: Stage-10
  893. Map Reduce Local Work
  894. Alias -> Map Local Tables:
  895. $INTNAME
  896. Fetch Operator
  897. limit: -1
  898. $INTNAME2
  899. Fetch Operator
  900. limit: -1
  901. Alias -> Map Local Operator Tree:
  902. $INTNAME
  903. HashTable Sink Operator
  904. condition expressions:
  905. 0
  906. 1 {_col0} {_col1}
  907. 2
  908. handleSkewJoin: false
  909. keys:
  910. 0 [Column[_col0]]
  911. 1 [Column[_col0]]
  912. 2 [Column[_col0]]
  913. Position of Big Table: 0
  914. $INTNAME2
  915. HashTable Sink Operator
  916. condition expressions:
  917. 0
  918. 1 {_col0} {_col1}
  919. 2
  920. handleSkewJoin: false
  921. keys:
  922. 0 [Column[_col0]]
  923. 1 [Column[_col0]]
  924. 2 [Column[_col0]]
  925. Position of Big Table: 0
  926. Stage: Stage-6
  927. Map Reduce
  928. Alias -> Map Operator Tree:
  929. $INTNAME1
  930. Map Join Operator
  931. condition map:
  932. Inner Join 0 to 1
  933. Inner Join 0 to 2
  934. condition expressions:
  935. 0
  936. 1 {_col0} {_col1}
  937. 2
  938. handleSkewJoin: false
  939. keys:
  940. 0 [Column[_col0]]
  941. 1 [Column[_col0]]
  942. 2 [Column[_col0]]
  943. outputColumnNames: _col2, _col3
  944. Position of Big Table: 0
  945. Select Operator
  946. expressions:
  947. expr: _col2
  948. type: string
  949. expr: _col3
  950. type: string
  951. outputColumnNames: _col2, _col3
  952. Group By Operator
  953. aggregations:
  954. expr: sum(hash(_col2,_col3))
  955. bucketGroup: false
  956. mode: hash
  957. outputColumnNames: _col0
  958. File Output Operator
  959. compressed: false
  960. GlobalTableId: 0
  961. table:
  962. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  963. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  964. Local Work:
  965. Map Reduce Local Work
  966. Stage: Stage-3
  967. Map Reduce
  968. Alias -> Map Operator Tree:
  969. file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-31-13_566_5103248435836603181/-mr-10003
  970. Reduce Output Operator
  971. sort order:
  972. tag: -1
  973. value expressions:
  974. expr: _col0
  975. type: bigint
  976. Reduce Operator Tree:
  977. Group By Operator
  978. aggregations:
  979. expr: sum(VALUE._col0)
  980. bucketGroup: false
  981. mode: mergepartial
  982. outputColumnNames: _col0
  983. Select Operator
  984. expressions:
  985. expr: _col0
  986. type: bigint
  987. outputColumnNames: _col0
  988. File Output Operator
  989. compressed: false
  990. GlobalTableId: 0
  991. table:
  992. input format: org.apache.hadoop.mapred.TextInputFormat
  993. output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
  994. Stage: Stage-11
  995. Map Reduce Local Work
  996. Alias -> Map Local Tables:
  997. $INTNAME
  998. Fetch Operator
  999. limit: -1
  1000. $INTNAME1
  1001. Fetch Operator
  1002. limit: -1
  1003. Alias -> Map Local Operator Tree:
  1004. $INTNAME
  1005. HashTable Sink Operator
  1006. condition expressions:
  1007. 0
  1008. 1 {_col0} {_col1}
  1009. 2
  1010. handleSkewJoin: false
  1011. keys:
  1012. 0 [Column[_col0]]
  1013. 1 [Column[_col0]]
  1014. 2 [Column[_col0]]
  1015. Position of Big Table: 1
  1016. $INTNAME1
  1017. HashTable Sink Operator
  1018. condition expressions:
  1019. 0
  1020. 1 {_col0} {_col1}
  1021. 2
  1022. handleSkewJoin: false
  1023. keys:
  1024. 0 [Column[_col0]]
  1025. 1 [Column[_col0]]
  1026. 2 [Column[_col0]]
  1027. Position of Big Table: 1
  1028. Stage: Stage-7
  1029. Map Reduce
  1030. Alias -> Map Operator Tree:
  1031. $INTNAME2
  1032. Map Join Operator
  1033. condition map:
  1034. Inner Join 0 to 1
  1035. Inner Join 0 to 2
  1036. condition expressions:
  1037. 0
  1038. 1 {_col0} {_col1}
  1039. 2
  1040. handleSkewJoin: false
  1041. keys:
  1042. 0 [Column[_col0]]
  1043. 1 [Column[_col0]]
  1044. 2 [Column[_col0]]
  1045. outputColumnNames: _col2, _col3
  1046. Position of Big Table: 1
  1047. Select Operator
  1048. expressions:
  1049. expr: _col2
  1050. type: string
  1051. expr: _col3
  1052. type: string
  1053. outputColumnNames: _col2, _col3
  1054. Group By Operator
  1055. aggregations:
  1056. expr: sum(hash(_col2,_col3))
  1057. bucketGroup: false
  1058. mode: hash
  1059. outputColumnNames: _col0
  1060. File Output Operator
  1061. compressed: false
  1062. GlobalTableId: 0
  1063. table:
  1064. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  1065. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  1066. Local Work:
  1067. Map Reduce Local Work
  1068. Stage: Stage-12
  1069. Map Reduce Local Work
  1070. Alias -> Map Local Tables:
  1071. $INTNAME1
  1072. Fetch Operator
  1073. limit: -1
  1074. $INTNAME2
  1075. Fetch Operator
  1076. limit: -1
  1077. Alias -> Map Local Operator Tree:
  1078. $INTNAME1
  1079. HashTable Sink Operator
  1080. condition expressions:
  1081. 0
  1082. 1 {_col0} {_col1}
  1083. 2
  1084. handleSkewJoin: false
  1085. keys:
  1086. 0 [Column[_col0]]
  1087. 1 [Column[_col0]]
  1088. 2 [Column[_col0]]
  1089. Position of Big Table: 2
  1090. $INTNAME2
  1091. HashTable Sink Operator
  1092. condition expressions:
  1093. 0
  1094. 1 {_col0} {_col1}
  1095. 2
  1096. handleSkewJoin: false
  1097. keys:
  1098. 0 [Column[_col0]]
  1099. 1 [Column[_col0]]
  1100. 2 [Column[_col0]]
  1101. Position of Big Table: 2
  1102. Stage: Stage-8
  1103. Map Reduce
  1104. Alias -> Map Operator Tree:
  1105. $INTNAME
  1106. Map Join Operator
  1107. condition map:
  1108. Inner Join 0 to 1
  1109. Inner Join 0 to 2
  1110. condition expressions:
  1111. 0
  1112. 1 {_col0} {_col1}
  1113. 2
  1114. handleSkewJoin: false
  1115. keys:
  1116. 0 [Column[_col0]]
  1117. 1 [Column[_col0]]
  1118. 2 [Column[_col0]]
  1119. outputColumnNames: _col2, _col3
  1120. Position of Big Table: 2
  1121. Select Operator
  1122. expressions:
  1123. expr: _col2
  1124. type: string
  1125. expr: _col3
  1126. type: string
  1127. outputColumnNames: _col2, _col3
  1128. Group By Operator
  1129. aggregations:
  1130. expr: sum(hash(_col2,_col3))
  1131. bucketGroup: false
  1132. mode: hash
  1133. outputColumnNames: _col0
  1134. File Output Operator
  1135. compressed: false
  1136. GlobalTableId: 0
  1137. table:
  1138. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  1139. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  1140. Local Work:
  1141. Map Reduce Local Work
  1142. Stage: Stage-2
  1143. Map Reduce
  1144. Alias -> Map Operator Tree:
  1145. $INTNAME
  1146. Reduce Output Operator
  1147. key expressions:
  1148. expr: _col0
  1149. type: string
  1150. sort order: +
  1151. Map-reduce partition columns:
  1152. expr: _col0
  1153. type: string
  1154. tag: 2
  1155. $INTNAME1
  1156. Reduce Output Operator
  1157. key expressions:
  1158. expr: _col0
  1159. type: string
  1160. sort order: +
  1161. Map-reduce partition columns:
  1162. expr: _col0
  1163. type: string
  1164. tag: 0
  1165. $INTNAME2
  1166. Reduce Output Operator
  1167. key expressions:
  1168. expr: _col0
  1169. type: string
  1170. sort order: +
  1171. Map-reduce partition columns:
  1172. expr: _col0
  1173. type: string
  1174. tag: 1
  1175. value expressions:
  1176. expr: _col0
  1177. type: string
  1178. expr: _col1
  1179. type: string
  1180. Reduce Operator Tree:
  1181. Join Operator
  1182. condition map:
  1183. Inner Join 0 to 1
  1184. Inner Join 0 to 2
  1185. condition expressions:
  1186. 0
  1187. 1 {VALUE._col0} {VALUE._col1}
  1188. 2
  1189. handleSkewJoin: false
  1190. outputColumnNames: _col2, _col3
  1191. Select Operator
  1192. expressions:
  1193. expr: _col2
  1194. type: string
  1195. expr: _col3
  1196. type: string
  1197. outputColumnNames: _col2, _col3
  1198. Group By Operator
  1199. aggregations:
  1200. expr: sum(hash(_col2,_col3))
  1201. bucketGroup: false
  1202. mode: hash
  1203. outputColumnNames: _col0
  1204. File Output Operator
  1205. compressed: false
  1206. GlobalTableId: 0
  1207. table:
  1208. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  1209. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  1210. Stage: Stage-4
  1211. Map Reduce
  1212. Alias -> Map Operator Tree:
  1213. x:src
  1214. TableScan
  1215. alias: src
  1216. Select Operator
  1217. expressions:
  1218. expr: key
  1219. type: string
  1220. expr: value
  1221. type: string
  1222. outputColumnNames: _col0, _col1
  1223. Reduce Output Operator
  1224. key expressions:
  1225. expr: _col0
  1226. type: string
  1227. sort order: +
  1228. tag: -1
  1229. value expressions:
  1230. expr: _col0
  1231. type: string
  1232. expr: _col1
  1233. type: string
  1234. Reduce Operator Tree:
  1235. Extract
  1236. File Output Operator
  1237. compressed: false
  1238. GlobalTableId: 0
  1239. table:
  1240. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  1241. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  1242. Stage: Stage-5
  1243. Map Reduce
  1244. Alias -> Map Operator Tree:
  1245. y:src
  1246. TableScan
  1247. alias: src
  1248. Select Operator
  1249. expressions:
  1250. expr: key
  1251. type: string
  1252. expr: value
  1253. type: string
  1254. outputColumnNames: _col0, _col1
  1255. Reduce Output Operator
  1256. key expressions:
  1257. expr: _col1
  1258. type: string
  1259. sort order: +
  1260. tag: -1
  1261. value expressions:
  1262. expr: _col0
  1263. type: string
  1264. expr: _col1
  1265. type: string
  1266. Reduce Operator Tree:
  1267. Extract
  1268. File Output Operator
  1269. compressed: false
  1270. GlobalTableId: 0
  1271. table:
  1272. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  1273. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  1274. Stage: Stage-0
  1275. Fetch Operator
  1276. limit: -1
  1277. PREHOOK: query: FROM
  1278. (SELECT src.* FROM src sort by key) x
  1279. JOIN
  1280. (SELECT src.* FROM src sort by value) Y
  1281. ON (x.key = Y.key)
  1282. JOIN
  1283. (SELECT src.* FROM src sort by value) Z
  1284. ON (x.key = Z.key)
  1285. select sum(hash(Y.key,Y.value))
  1286. PREHOOK: type: QUERY
  1287. PREHOOK: Input: default@src
  1288. PREHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-31-14_078_677078805297192158/-mr-10000
  1289. POSTHOOK: query: FROM
  1290. (SELECT src.* FROM src sort by key) x
  1291. JOIN
  1292. (SELECT src.* FROM src sort by value) Y
  1293. ON (x.key = Y.key)
  1294. JOIN
  1295. (SELECT src.* FROM src sort by value) Z
  1296. ON (x.key = Z.key)
  1297. select sum(hash(Y.key,Y.value))
  1298. POSTHOOK: type: QUERY
  1299. POSTHOOK: Input: default@src
  1300. POSTHOOK: Output: file:/var/folders/6g/6grtCwPMEf4sqHUPpy6xQG9ByHg/-Tmp-/heyongqiang/hive_2011-04-07_01-31-14_078_677078805297192158/-mr-10000
  1301. 348019368476
  1302. PREHOOK: query: explain
  1303. FROM
  1304. (SELECT src.* FROM src sort by key) x
  1305. JOIN
  1306. (SELECT src.* FROM src sort by value) Y
  1307. ON (x.key = Y.key)
  1308. LEFT OUTER JOIN
  1309. (SELECT src.* FROM src sort by value) Z
  1310. ON (x.key = Z.key)
  1311. select sum(hash(Y.key,Y.value))
  1312. PREHOOK: type: QUERY
  1313. POSTHOOK: query: explain
  1314. FROM
  1315. (SELECT src.* FROM src sort by key) x
  1316. JOIN
  1317. (SELECT src.* FROM src sort by value) Y
  1318. ON (x.key = Y.key)
  1319. LEFT OUTER JOIN
  1320. (SELECT src.* FROM src sort by value) Z
  1321. ON (x.key = Z.key)
  1322. select sum(hash(Y.key,Y.value))
  1323. POSTHOOK: type: QUERY
  1324. ABSTRACT SYNTAX TREE:
  1325. (TOK_QUERY (TOK_FROM (TOK_LEFTOUTERJOIN (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key))))) x) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Y) key))) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME src)))) (TOK_SORTBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) Z) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL Z) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION sum (TOK_FUNCTION hash (. (TOK_TABLE_OR_COL Y) key) (. (TOK_TABLE_OR_COL Y) value)))))))
  1326. STAGE DEPENDENCIES:
  1327. Stage-1 is a root stage
  1328. Stage-8 depends on stages: Stage-1, Stage-4, Stage-5 , consists of Stage-9, Stage-10, Stage-2
  1329. Stage-9 has a backup stage: Stage-2
  1330. Stage-6 depends on stages: Stage-9
  1331. Stage-3 depends on stages: Stage-2, Stage-6, Stage-7
  1332. Stage-10 has a backup stage: Stage-2
  1333. Stage-7 depends on stages: Stage-10
  1334. Stage-2
  1335. Stage-4 is a root stage
  1336. Stage-5 is a root stage
  1337. Stage-0 is a root stage
  1338. STAGE PLANS:
  1339. Stage: Stage-1
  1340. Map Reduce
  1341. Alias -> Map Operator Tree:
  1342. z:src
  1343. TableScan
  1344. alias: src
  1345. Select Operator
  1346. expressions:
  1347. expr: key
  1348. type: string
  1349. expr: value
  1350. type: string
  1351. outputColumnNames: _col0, _col1
  1352. Reduce Output Operator
  1353. key expressions:
  1354. expr: _col1
  1355. type: string
  1356. sort order: +
  1357. tag: -1
  1358. value expressions:
  1359. expr: _col0
  1360. type: string
  1361. expr: _col1
  1362. type: string
  1363. Reduce Operator Tree:
  1364. Extract
  1365. File Output Operator
  1366. compressed: false
  1367. GlobalTableId: 0
  1368. table:
  1369. input format: org.apache.hadoop.mapred.SequenceFileInputFormat
  1370. output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
  1371. Stage: Stage-8
  1372. Conditional Operator
  1373. Stage: Stage-9
  1374. Map Reduce Local Work
  1375. Alias -> Map Local Tables:
  1376. $INTNAME
  1377. Fetch Operator
  1378. limit: -1
  1379. $INTNAME2
  1380. Fetch Operator
  1381. limit: -1
  1382. Alias -> Map Local Operator Tree:
  1383. $INTNAME
  1384. HashTable Sink Operator
  1385. condition expressions:
  1386. 0
  1387. 1 {_col0} {_col1}
  1388. 2
  1389. handleSkewJoin: false
  1390. keys:
  1391. 0 [Column[_col0]]
  1392. 1 [Column[_col0]]
  1393. 2 [Column[_col0]]
  1394. Position of Big Table: 0
  1395. $INTNAME2
  1396. HashTable Sink Operator
  1397. condition expressions:
  1398. 0
  1399. 1 {_col0} {_col1}
  1400. 2
  1401. handleSkewJoin: false
  1402. keys:
  1403. 0 [Column[_col0]]
  1404. 1 [Column[_col0]]
  1405. 2 [Column[_col0]]
  1406. Position of Big Table: 0
  1407. Stage: Stage-6

Large files files are truncated, but you can click here to view the full file