== Physical Plan ==
AdaptiveSparkPlan (849)
+- == Final Plan ==
ResultQueryStage (844), Statistics(sizeInBytes=8.0 EiB)
+- Execute InsertIntoHadoopFsRelationCommand (843)
+- WriteFiles (842)
+- Project (841)
+- Filter (840)
+- TableCacheQueryStage (839), Statistics(sizeInBytes=10.6 KiB, rowCount=41)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- AdaptiveSparkPlan (838)
+- == Final Plan ==
ResultQueryStage (828)
+- * Filter (827)
+- ObjectHashAggregate (826)
+- ShuffleQueryStage (825), Statistics(sizeInBytes=13.0 KiB, rowCount=41)
+- Exchange (824)
+- ObjectHashAggregate (823)
+- Project (822)
+- Window (821)
+- * Sort (820)
+- AQEShuffleRead (819)
+- ShuffleQueryStage (818), Statistics(sizeInBytes=17.7 KiB, rowCount=41)
+- Exchange (817)
+- * Project (816)
+- TableCacheQueryStage (815), Statistics(sizeInBytes=21.9 KiB, rowCount=41)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildRight (256)
: : : :- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- BroadcastQueryStage (255), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (254)
: : : +- AQEShuffleRead (253)
: : : +- ShuffleQueryStage (252), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (251)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
+- == Initial Plan ==
Filter (837)
+- ObjectHashAggregate (836)
+- Exchange (835)
+- ObjectHashAggregate (834)
+- Project (833)
+- Window (832)
+- Sort (831)
+- Exchange (830)
+- Project (829)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildRight (256)
: : : :- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- BroadcastQueryStage (255), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (254)
: : : +- AQEShuffleRead (253)
: : : +- ShuffleQueryStage (252), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (251)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
+- == Initial Plan ==
Execute InsertIntoHadoopFsRelationCommand (848)
+- WriteFiles (847)
+- Project (846)
+- Filter (845)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- AdaptiveSparkPlan (838)
+- == Final Plan ==
ResultQueryStage (828)
+- * Filter (827)
+- ObjectHashAggregate (826)
+- ShuffleQueryStage (825), Statistics(sizeInBytes=13.0 KiB, rowCount=41)
+- Exchange (824)
+- ObjectHashAggregate (823)
+- Project (822)
+- Window (821)
+- * Sort (820)
+- AQEShuffleRead (819)
+- ShuffleQueryStage (818), Statistics(sizeInBytes=17.7 KiB, rowCount=41)
+- Exchange (817)
+- * Project (816)
+- TableCacheQueryStage (815), Statistics(sizeInBytes=21.9 KiB, rowCount=41)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildRight (256)
: : : :- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- BroadcastQueryStage (255), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (254)
: : : +- AQEShuffleRead (253)
: : : +- ShuffleQueryStage (252), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (251)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
+- == Initial Plan ==
Filter (837)
+- ObjectHashAggregate (836)
+- Exchange (835)
+- ObjectHashAggregate (834)
+- Project (833)
+- Window (832)
+- Sort (831)
+- Exchange (830)
+- Project (829)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildRight (256)
: : : :- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- BroadcastQueryStage (255), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (254)
: : : +- AQEShuffleRead (253)
: : : +- ShuffleQueryStage (252), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (251)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
(1) InMemoryTableScan
Output [3]: [openGoodsMovements#145382, plant#143832, product#143833]
Arguments: [openGoodsMovements#145382, plant#143832, product#143833], [(size(transform(openGoodsMovements#145382, lambdafunction(struct(documentType, PO, documentId, lambda x_42#153457.documentId, documentUuid, lambda x_42#153457.documentUuid, openQty, lambda x_42#153457.openQty, qtyUomUuid, lambda x_42#153457.qtyUomUuid, qtyUom, lambda x_42#153457.qtyUom, plannedDeliveryDatetime, lambda x_42#153457.availabilityDatetime), lambda x_42#153457, false)), false) > 0)]
(2) InMemoryRelation
Arguments: [product#143833, plant#143832, openGoodsMovements#145382], StorageLevel(disk, memory, 2 replicas)
(3) InMemoryTableScan
Output [10]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityDenominator#143840, quantityNumerator#143839, quantityUom#143837, quantityUomUuid#143838, scheduleLines#143841, validConfirmations#143932]
Arguments: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityDenominator#143840, quantityNumerator#143839, quantityUom#143837, quantityUomUuid#143838, scheduleLines#143841, validConfirmations#143932]
(4) InMemoryRelation
Arguments: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842, stockDatetime#143384, timezonecode#143012, validConfirmations#143932], StorageLevel(disk, memory, deserialized, 1 replicas)
(5) Scan parquet
Output [3]: [product#142980, plant#142981, orderDateTime#142982]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(6) ColumnarToRow [codegen id : 1]
Input [3]: [product#142980, plant#142981, orderDateTime#142982]
(7) Filter [codegen id : 1]
Input [3]: [product#142980, plant#142981, orderDateTime#142982]
Condition : (isnotnull(plant#142981) AND isnotnull(product#142980))
(8) Exchange
Input [3]: [product#142980, plant#142981, orderDateTime#142982]
Arguments: hashpartitioning(product#142980, plant#142981, 25), REPARTITION_BY_NUM, [plan_id=226365]
(9) ShuffleQueryStage
Output [3]: [product#142980, plant#142981, orderDateTime#142982]
Arguments: 0
(10) Scan parquet
Output [4]: [plant#142987, product#142988, productionAspect#142990, systemId#142985]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(11) ColumnarToRow [codegen id : 2]
Input [4]: [plant#142987, product#142988, productionAspect#142990, systemId#142985]
(12) Project [codegen id : 2]
Output [4]: [plant#142987.internalRefUUID AS _extract_internalRefUUID#144131, product#142988.internalRefUUID AS _extract_internalRefUUID#144130, productionAspect#142990.productMovementPlants.unitOfIssue.code AS _extract_code#144128, productionAspect#142990.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144129]
Input [4]: [plant#142987, product#142988, productionAspect#142990, systemId#142985]
(13) Exchange
Input [4]: [_extract_internalRefUUID#144131, _extract_internalRefUUID#144130, _extract_code#144128, _extract_internalRefUUID#144129]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226381]
(14) ShuffleQueryStage
Output [4]: [_extract_internalRefUUID#144131, _extract_internalRefUUID#144130, _extract_code#144128, _extract_internalRefUUID#144129]
Arguments: 1
(15) Project [codegen id : 61]
Output [4]: [_extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
Input [4]: [_extract_internalRefUUID#144131, _extract_internalRefUUID#144130, _extract_code#144128, _extract_internalRefUUID#144129]
(16) BroadcastExchange
Input [4]: [_extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=236366]
(17) BroadcastQueryStage
Output [4]: [_extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
Arguments: 122
(18) BroadcastHashJoin [codegen id : 102]
Left keys [4]: [coalesce(product#142980, ), isnull(product#142980), coalesce(plant#142981, ), isnull(plant#142981)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144130, ), isnull(_extract_internalRefUUID#144130), coalesce(_extract_internalRefUUID#144131, ), isnull(_extract_internalRefUUID#144131)]
Join type: LeftOuter
Join condition: None
(19) Project [codegen id : 102]
Output [5]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129]
Input [7]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
(20) Scan parquet
Output [3]: [internalUUID#142998, address#143004, systemId#142997]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string,address:struct<timeZone:struct<code:string>>>
(21) ColumnarToRow [codegen id : 3]
Input [3]: [internalUUID#142998, address#143004, systemId#142997]
(22) Project [codegen id : 3]
Output [2]: [internalUUID#142998, address#143004.timeZone.code AS timezonecode#143009]
Input [3]: [internalUUID#142998, address#143004, systemId#142997]
(23) Exchange
Input [2]: [internalUUID#142998, timezonecode#143009]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226400]
(24) ShuffleQueryStage
Output [2]: [internalUUID#142998, timezonecode#143009]
Arguments: 2
(25) BroadcastExchange
Input [2]: [internalUUID#142998, timezonecode#143009]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=234017]
(26) BroadcastQueryStage
Output [2]: [internalUUID#142998, timezonecode#143009]
Arguments: 120
(27) BroadcastHashJoin [codegen id : 102]
Left keys [2]: [coalesce(plant#142981, ), isnull(plant#142981)]
Right keys [2]: [coalesce(internalUUID#142998, ), isnull(internalUUID#142998)]
Join type: LeftOuter
Join condition: None
(28) Project [codegen id : 102]
Output [6]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, CASE WHEN isnotnull(timezonecode#143009) THEN timezonecode#143009 ELSE UTC END AS timezonecode#143012]
Input [7]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, internalUUID#142998, timezonecode#143009]
(29) Scan parquet
Output [4]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020, systemId#143013]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(30) ColumnarToRow [codegen id : 4]
Input [4]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020, systemId#143013]
(31) Filter [codegen id : 4]
Input [4]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020, systemId#143013]
Condition : isnotnull(internalUUID#143014)
(32) Project [codegen id : 4]
Output [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Input [4]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020, systemId#143013]
(33) Exchange
Input [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226420]
(34) ShuffleQueryStage
Output [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Arguments: 3
(35) BroadcastExchange
Input [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=228578]
(36) BroadcastQueryStage
Output [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Arguments: 97
(37) BroadcastHashJoin [codegen id : 102]
Left keys [1]: [product#142980]
Right keys [1]: [internalUUID#143014]
Join type: LeftOuter
Join condition: None
(38) Project
Output [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filter(unitOfMeasureConversions#143020, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144128) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144129)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143018.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143018.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143026]
Input [9]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, timezonecode#143012, internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
(39) Generate [codegen id : 103]
Input [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026]
Arguments: explode(filteredUnitOfMeasureConversions#143026.quantityNumerator), [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026], true, [quantityNumerator#143034]
(40) Project [codegen id : 103]
Output [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026.quantityDenominator AS _extract_quantityDenominator#143934]
Input [6]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026, quantityNumerator#143034]
(41) Generate [codegen id : 103]
Input [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, _extract_quantityDenominator#143934]
Arguments: explode(_extract_quantityDenominator#143934), [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012], true, [quantityDenominator#143037]
(42) Project [codegen id : 103]
Output [4]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012]
Input [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, quantityDenominator#143037]
(43) Scan parquet
Output [6]: [internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, systemId#143076]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(44) ColumnarToRow [codegen id : 5]
Input [6]: [internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, systemId#143076]
(45) Filter [codegen id : 5]
Input [6]: [internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, systemId#143076]
Condition : (isnotnull(plant#143079.internalRefUUID) AND isnotnull(product#143078.internalRefUUID))
(46) Project [codegen id : 5]
Output [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Input [6]: [internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, systemId#143076]
(47) Exchange
Input [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226447]
(48) ShuffleQueryStage
Output [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Arguments: 4
(49) BroadcastExchange
Input [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=232392]
(50) BroadcastQueryStage
Output [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Arguments: 116
(51) Scan parquet
Output [2]: [marketUnit#143040, distributionCenters#143046]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(52) ColumnarToRow [codegen id : 6]
Input [2]: [marketUnit#143040, distributionCenters#143046]
(53) Filter [codegen id : 6]
Input [2]: [marketUnit#143040, distributionCenters#143046]
Condition : (((marketUnit#143040 <=> Daily_grocery) AND (size(distributionCenters#143046, true) > 0)) AND isnotnull(distributionCenters#143046))
(54) Project [codegen id : 6]
Output [1]: [distributionCenters#143046]
Input [2]: [marketUnit#143040, distributionCenters#143046]
(55) Exchange
Input [1]: [distributionCenters#143046]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226467]
(56) ShuffleQueryStage
Output [1]: [distributionCenters#143046]
Arguments: 5
(57) Generate [codegen id : 56]
Input [1]: [distributionCenters#143046]
Arguments: explode(distributionCenters#143046), false, [distributionCenters#143064]
(58) Filter [codegen id : 56]
Input [1]: [distributionCenters#143064]
Condition : (((size(distributionCenters#143064.storageLocations, true) > 0) AND isnotnull(distributionCenters#143064.storageLocations)) AND isnotnull(distributionCenters#143064.internalRefUUID))
(59) Project [codegen id : 56]
Output [2]: [distributionCenters#143064.internalRefUUID AS _extract_internalRefUUID#143938, distributionCenters#143064.storageLocations AS _extract_storageLocations#143939]
Input [1]: [distributionCenters#143064]
(60) Generate [codegen id : 56]
Input [2]: [_extract_internalRefUUID#143938, _extract_storageLocations#143939]
Arguments: explode(_extract_storageLocations#143939), [_extract_internalRefUUID#143938], false, [storageLocations#143071]
(61) Project [codegen id : 56]
Output [3]: [_extract_internalRefUUID#143938 AS plantUuid#143066, storageLocations#143071.id AS storageLocId#143072, storageLocations#143071.internalRefUUID AS storageLocUuid#143073]
Input [2]: [_extract_internalRefUUID#143938, storageLocations#143071]
(62) BroadcastExchange
Input [3]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=231359]
(63) BroadcastQueryStage
Output [3]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Arguments: 108
(64) Scan parquet
Output [2]: [product#143336, plant#143337]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(65) ColumnarToRow [codegen id : 7]
Input [2]: [product#143336, plant#143337]
(66) Filter [codegen id : 7]
Input [2]: [product#143336, plant#143337]
Condition : isnotnull(plant#143337)
(67) Exchange
Input [2]: [product#143336, plant#143337]
Arguments: hashpartitioning(product#143336, plant#143337, 25), REPARTITION_BY_NUM, [plan_id=226490]
(68) ShuffleQueryStage
Output [2]: [product#143336, plant#143337]
Arguments: 6
(69) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#144134, _extract_internalRefUUID#144135, _extract_internalRefUUID#144132, _extract_internalRefUUID#144133]
(70) BroadcastQueryStage
Output [4]: [_extract_code#144134, _extract_internalRefUUID#144135, _extract_internalRefUUID#144132, _extract_internalRefUUID#144133]
Arguments: 124
(71) BroadcastHashJoin [codegen id : 73]
Left keys [4]: [coalesce(product#143336, ), isnull(product#143336), coalesce(plant#143337, ), isnull(plant#143337)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144132, ), isnull(_extract_internalRefUUID#144132), coalesce(_extract_internalRefUUID#144133, ), isnull(_extract_internalRefUUID#144133)]
Join type: LeftOuter
Join condition: None
(72) Project [codegen id : 73]
Output [4]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135]
Input [6]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135, _extract_internalRefUUID#144132, _extract_internalRefUUID#144133]
(73) Scan parquet
Output [2]: [internalUUID#143350, systemId#143349]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(74) ColumnarToRow [codegen id : 9]
Input [2]: [internalUUID#143350, systemId#143349]
(75) Project [codegen id : 9]
Output [1]: [internalUUID#143350]
Input [2]: [internalUUID#143350, systemId#143349]
(76) Exchange
Input [1]: [internalUUID#143350]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226527]
(77) ShuffleQueryStage
Output [1]: [internalUUID#143350]
Arguments: 9
(78) BroadcastExchange
Input [1]: [internalUUID#143350]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=237446]
(79) BroadcastQueryStage
Output [1]: [internalUUID#143350]
Arguments: 141
(80) BroadcastHashJoin [codegen id : 73]
Left keys [2]: [coalesce(plant#143337, ), isnull(plant#143337)]
Right keys [2]: [coalesce(internalUUID#143350, ), isnull(internalUUID#143350)]
Join type: LeftOuter
Join condition: None
(81) Project [codegen id : 73]
Output [4]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135]
Input [5]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135, internalUUID#143350]
(82) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
(83) BroadcastQueryStage
Output [3]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
Arguments: 98
(84) BroadcastHashJoin [codegen id : 73]
Left keys [1]: [product#143336]
Right keys [1]: [internalUUID#143364]
Join type: LeftOuter
Join condition: None
(85) Project
Output [2]: [plant#143337, filter(unitOfMeasureConversions#143370, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144134) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144135)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143368.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143368.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143375]
Input [7]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135, internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
(86) Generate [codegen id : 74]
Input [2]: [plant#143337, filteredUnitOfMeasureConversions#143375]
Arguments: explode(filteredUnitOfMeasureConversions#143375.quantityNumerator), [plant#143337, filteredUnitOfMeasureConversions#143375], true, [quantityNumerator#143376]
(87) Project [codegen id : 74]
Output [2]: [plant#143337, filteredUnitOfMeasureConversions#143375.quantityDenominator AS _extract_quantityDenominator#143940]
Input [3]: [plant#143337, filteredUnitOfMeasureConversions#143375, quantityNumerator#143376]
(88) Generate [codegen id : 74]
Input [2]: [plant#143337, _extract_quantityDenominator#143940]
Arguments: explode(_extract_quantityDenominator#143940), [plant#143337], true, [quantityDenominator#143377]
(89) Project [codegen id : 74]
Output [1]: [plant#143337]
Input [2]: [plant#143337, quantityDenominator#143377]
(90) HashAggregate [codegen id : 74]
Input [1]: [plant#143337]
Keys [1]: [plant#143337]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143337]
(91) Exchange
Input [1]: [plant#143337]
Arguments: hashpartitioning(plant#143337, 25), ENSURE_REQUIREMENTS, [plan_id=254858]
(92) ShuffleQueryStage
Output [1]: [plant#143337]
Arguments: 152
(93) AQEShuffleRead
Input [1]: [plant#143337]
Arguments: coalesced
(94) HashAggregate
Input [1]: [plant#143337]
Keys [1]: [plant#143337]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143337]
(95) BroadcastHashJoin
Left keys [1]: [plantUuid#143066]
Right keys [1]: [plant#143337]
Join type: Inner
Join condition: None
(96) Project
Output [3]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Input [4]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073, plant#143337]
(97) BroadcastHashJoin [codegen id : 83]
Left keys [2]: [coalesce(plant#143079.internalRefUUID, ), isnull(plant#143079.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#143066, ), isnull(plantUuid#143066)]
Join type: Inner
Join condition: None
(98) Generate
Input [9]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Arguments: explode(filter(stockQuantities#143081, lambdafunction(((lambda x#143085.storageLocation.internalRefUuid = storageLocUuid#143073) AND (lambda x#143085.storageLocation.Id = storageLocId#143072)), lambda x#143085, false))), [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, plantUuid#143066], false, [stockQuantity#143086]
(99) ObjectHashAggregate
Input [7]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, plantUuid#143066, stockQuantity#143086]
Keys [5]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076]
Functions [2]: [partial_first(calculationDateTime#143080, false), partial_collect_list(stockQuantity#143086, 0, 0)]
Aggregate Attributes [3]: [first#144812, valueSet#144813, buf#144814]
Results [8]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076, first#144815, valueSet#144816, buf#144817]
(100) ObjectHashAggregate
Input [8]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076, first#144815, valueSet#144816, buf#144817]
Keys [5]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076]
Functions [2]: [first(calculationDateTime#143080, false), collect_list(stockQuantity#143086, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#143080)()#143100, collect_list(stockQuantity#143086, 0, 0)#143101]
Results [4]: [product#143078, plant#143079, first(calculationDateTime#143080)()#143100 AS calculationDateTime#143087, collect_list(stockQuantity#143086, 0, 0)#143101 AS stockQuantities#143088]
(101) Filter [codegen id : 84]
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantities#143088]
Condition : ((size(stockQuantities#143088, true) > 0) AND isnotnull(calculationDateTime#143087))
(102) Generate [codegen id : 84]
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantities#143088]
Arguments: explode(stockQuantities#143088), [product#143078, plant#143079, calculationDateTime#143087], false, [stockQuantity#143103]
(103) Filter [codegen id : 84]
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantity#143103]
Condition : (((isnotnull(stockQuantity#143103.specialStockIndicator.code) AND isnotnull(stockQuantity#143103.stockType.code)) AND (stockQuantity#143103.specialStockIndicator.code = )) AND (stockQuantity#143103.stockType.code = 01))
(104) Project [codegen id : 84]
Output [4]: [plant#143079.internalRefUUID AS plantUuid#143105, product#143078.internalRefUUID AS productUuid#143107, calculationDateTime#143087, stockQuantity#143103.storagelocation.id AS storageLocId#143108]
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantity#143103]
(105) Exchange
Input [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: hashpartitioning(productUuid#143107, plantUuid#143105, 25), ENSURE_REQUIREMENTS, [plan_id=257985]
(106) ShuffleQueryStage
Output [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: 161
(107) AQEShuffleRead
Input [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: local
(108) BroadcastExchange
Input [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true], input[1, string, true]),false), [plan_id=258936]
(109) BroadcastQueryStage
Output [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: 167
(110) Scan parquet
Output [3]: [product#143122, plant#143123, orderDateTime#143124]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(111) ColumnarToRow [codegen id : 11]
Input [3]: [product#143122, plant#143123, orderDateTime#143124]
(112) Filter [codegen id : 11]
Input [3]: [product#143122, plant#143123, orderDateTime#143124]
Condition : ((isnotnull(orderDateTime#143124) AND isnotnull(plant#143123)) AND isnotnull(product#143122))
(113) Exchange
Input [3]: [product#143122, plant#143123, orderDateTime#143124]
Arguments: hashpartitioning(product#143122, plant#143123, 25), REPARTITION_BY_NUM, [plan_id=226588]
(114) ShuffleQueryStage
Output [3]: [product#143122, plant#143123, orderDateTime#143124]
Arguments: 12
(115) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#144138, _extract_internalRefUUID#144139, _extract_internalRefUUID#144137, _extract_internalRefUUID#144136]
(116) BroadcastQueryStage
Output [4]: [_extract_code#144138, _extract_internalRefUUID#144139, _extract_internalRefUUID#144137, _extract_internalRefUUID#144136]
Arguments: 126
(117) BroadcastHashJoin [codegen id : 91]
Left keys [4]: [coalesce(product#143122, ), isnull(product#143122), coalesce(plant#143123, ), isnull(plant#143123)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144137, ), isnull(_extract_internalRefUUID#144137), coalesce(_extract_internalRefUUID#144136, ), isnull(_extract_internalRefUUID#144136)]
Join type: LeftOuter
Join condition: None
(118) Project [codegen id : 91]
Output [5]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139]
Input [7]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139, _extract_internalRefUUID#144137, _extract_internalRefUUID#144136]
(119) ReusedExchange [Reuses operator id: 78]
Output [1]: [internalUUID#143136]
(120) BroadcastQueryStage
Output [1]: [internalUUID#143136]
Arguments: 142
(121) BroadcastHashJoin [codegen id : 91]
Left keys [2]: [coalesce(plant#143123, ), isnull(plant#143123)]
Right keys [2]: [coalesce(internalUUID#143136, ), isnull(internalUUID#143136)]
Join type: LeftOuter
Join condition: None
(122) Project [codegen id : 91]
Output [5]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139]
Input [6]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139, internalUUID#143136]
(123) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
(124) BroadcastQueryStage
Output [3]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
Arguments: 99
(125) BroadcastHashJoin [codegen id : 91]
Left keys [1]: [product#143122]
Right keys [1]: [internalUUID#143150]
Join type: LeftOuter
Join condition: None
(126) Project
Output [4]: [product#143122, plant#143123, orderDateTime#143124, filter(unitOfMeasureConversions#143156, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144138) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144139)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143154.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143154.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143161]
Input [8]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139, internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
(127) Generate
Input [4]: [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161]
Arguments: explode(filteredUnitOfMeasureConversions#143161.quantityNumerator), [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161], true, [quantityNumerator#143162]
(128) Project
Output [4]: [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161.quantityDenominator AS _extract_quantityDenominator#143944]
Input [5]: [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161, quantityNumerator#143162]
(129) Generate
Input [4]: [product#143122, plant#143123, orderDateTime#143124, _extract_quantityDenominator#143944]
Arguments: explode(_extract_quantityDenominator#143944), [product#143122, plant#143123, orderDateTime#143124], true, [quantityDenominator#143163]
(130) Project
Output [3]: [product#143122, plant#143123, orderDateTime#143124]
Input [4]: [product#143122, plant#143123, orderDateTime#143124, quantityDenominator#143163]
(131) BroadcastHashJoin [codegen id : 92]
Left keys [2]: [plantUuid#143105, productUuid#143107]
Right keys [2]: [plant#143123, product#143122]
Join type: Inner
Join condition: (calculationDateTime#143087 <= orderDateTime#143124)
(132) Project [codegen id : 92]
Output [4]: [plantUuid#143105 AS plant#143170, productUuid#143107 AS product#143171, storageLocId#143108, calculationDateTime#143087]
Input [7]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108, product#143122, plant#143123, orderDateTime#143124]
(133) Exchange
Input [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: hashpartitioning(coalesce(plant#143170, ), isnull(plant#143170), coalesce(product#143171, ), isnull(product#143171), coalesce(storageLocId#143108, ), isnull(storageLocId#143108), coalesce(calculationDateTime#143087, 1970-01-01 00:00:00), isnull(calculationDateTime#143087), 25), ENSURE_REQUIREMENTS, [plan_id=259674]
(134) ShuffleQueryStage
Output [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: 172
(135) AQEShuffleRead
Input [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: local
(136) BroadcastExchange
Input [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true]), coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, timestamp, true], 1970-01-01 00:00:00), isnull(input[3, timestamp, true])),false), [plan_id=260723]
(137) BroadcastQueryStage
Output [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: 177
(138) ReusedExchange [Reuses operator id: 108]
Output [4]: [plantUuid#143262, productUuid#143264, calculationDateTime#143258, storageLocId#143265]
(139) BroadcastQueryStage
Output [4]: [plantUuid#143262, productUuid#143264, calculationDateTime#143258, storageLocId#143265]
Arguments: 169
(140) ReusedExchange [Reuses operator id: 113]
Output [3]: [product#143270, plant#143271, orderDateTime#143272]
(141) ShuffleQueryStage
Output [3]: [product#143270, plant#143271, orderDateTime#143272]
Arguments: 32
(142) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#144144, _extract_internalRefUUID#144145, _extract_internalRefUUID#144146, _extract_internalRefUUID#144147]
(143) BroadcastQueryStage
Output [4]: [_extract_code#144144, _extract_internalRefUUID#144145, _extract_internalRefUUID#144146, _extract_internalRefUUID#144147]
Arguments: 130
(144) BroadcastHashJoin [codegen id : 93]
Left keys [4]: [coalesce(product#143270, ), isnull(product#143270), coalesce(plant#143271, ), isnull(plant#143271)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144146, ), isnull(_extract_internalRefUUID#144146), coalesce(_extract_internalRefUUID#144147, ), isnull(_extract_internalRefUUID#144147)]
Join type: LeftOuter
Join condition: None
(145) Project [codegen id : 93]
Output [5]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145]
Input [7]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145, _extract_internalRefUUID#144146, _extract_internalRefUUID#144147]
(146) ReusedExchange [Reuses operator id: 78]
Output [1]: [internalUUID#143284]
(147) BroadcastQueryStage
Output [1]: [internalUUID#143284]
Arguments: 144
(148) BroadcastHashJoin [codegen id : 93]
Left keys [2]: [coalesce(plant#143271, ), isnull(plant#143271)]
Right keys [2]: [coalesce(internalUUID#143284, ), isnull(internalUUID#143284)]
Join type: LeftOuter
Join condition: None
(149) Project [codegen id : 93]
Output [5]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145]
Input [6]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145, internalUUID#143284]
(150) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
(151) BroadcastQueryStage
Output [3]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
Arguments: 101
(152) BroadcastHashJoin [codegen id : 93]
Left keys [1]: [product#143270]
Right keys [1]: [internalUUID#143298]
Join type: LeftOuter
Join condition: None
(153) Project
Output [4]: [product#143270, plant#143271, orderDateTime#143272, filter(unitOfMeasureConversions#143304, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144144) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144145)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143302.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143302.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143309]
Input [8]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145, internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
(154) Generate
Input [4]: [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309]
Arguments: explode(filteredUnitOfMeasureConversions#143309.quantityNumerator), [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309], true, [quantityNumerator#143310]
(155) Project
Output [4]: [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309.quantityDenominator AS _extract_quantityDenominator#143954]
Input [5]: [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309, quantityNumerator#143310]
(156) Generate
Input [4]: [product#143270, plant#143271, orderDateTime#143272, _extract_quantityDenominator#143954]
Arguments: explode(_extract_quantityDenominator#143954), [product#143270, plant#143271, orderDateTime#143272], true, [quantityDenominator#143311]
(157) Project
Output [3]: [product#143270, plant#143271, orderDateTime#143272]
Input [4]: [product#143270, plant#143271, orderDateTime#143272, quantityDenominator#143311]
(158) BroadcastHashJoin [codegen id : 94]
Left keys [2]: [plantUuid#143262, productUuid#143264]
Right keys [2]: [plant#143271, product#143270]
Join type: Inner
Join condition: (calculationDateTime#143258 <= orderDateTime#143272)
(159) Project [codegen id : 94]
Output [4]: [plantUuid#143262 AS plant#143314, productUuid#143264 AS product#143315, storageLocId#143265, calculationDateTime#143258]
Input [7]: [plantUuid#143262, productUuid#143264, calculationDateTime#143258, storageLocId#143265, product#143270, plant#143271, orderDateTime#143272]
(160) HashAggregate [codegen id : 94]
Input [4]: [plant#143314, product#143315, storageLocId#143265, calculationDateTime#143258]
Keys [3]: [plant#143314, product#143315, storageLocId#143265]
Functions [1]: [partial_max(calculationDateTime#143258)]
Aggregate Attributes [1]: [max#144818]
Results [4]: [plant#143314, product#143315, storageLocId#143265, max#144819]
(161) HashAggregate [codegen id : 94]
Input [4]: [plant#143314, product#143315, storageLocId#143265, max#144819]
Keys [3]: [plant#143314, product#143315, storageLocId#143265]
Functions [1]: [max(calculationDateTime#143258)]
Aggregate Attributes [1]: [max(calculationDateTime#143258)#143178]
Results [4]: [plant#143314, product#143315, storageLocId#143265, max(calculationDateTime#143258)#143178 AS max_calc_datetime#143172]
(162) Exchange
Input [4]: [plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
Arguments: hashpartitioning(coalesce(plant#143314, ), isnull(plant#143314), coalesce(product#143315, ), isnull(product#143315), coalesce(storageLocId#143265, ), isnull(storageLocId#143265), coalesce(max_calc_datetime#143172, 1970-01-01 00:00:00), isnull(max_calc_datetime#143172), 25), ENSURE_REQUIREMENTS, [plan_id=259733]
(163) ShuffleQueryStage
Output [4]: [plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
Arguments: 173
(164) AQEShuffleRead
Input [4]: [plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
Arguments: local
(165) BroadcastHashJoin [codegen id : 99]
Left keys [8]: [coalesce(plant#143170, ), isnull(plant#143170), coalesce(product#143171, ), isnull(product#143171), coalesce(storageLocId#143108, ), isnull(storageLocId#143108), coalesce(calculationDateTime#143087, 1970-01-01 00:00:00), isnull(calculationDateTime#143087)]
Right keys [8]: [coalesce(plant#143314, ), isnull(plant#143314), coalesce(product#143315, ), isnull(product#143315), coalesce(storageLocId#143265, ), isnull(storageLocId#143265), coalesce(max_calc_datetime#143172, 1970-01-01 00:00:00), isnull(max_calc_datetime#143172)]
Join type: Inner
Join condition: None
(166) Project [codegen id : 99]
Output [3]: [plant#143170, product#143171, calculationDateTime#143087]
Input [8]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087, plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
(167) HashAggregate [codegen id : 99]
Input [3]: [plant#143170, product#143171, calculationDateTime#143087]
Keys [2]: [product#143171, plant#143170]
Functions [1]: [partial_min(calculationDateTime#143087)]
Aggregate Attributes [1]: [min#144810]
Results [3]: [product#143171, plant#143170, min#144811]
(168) Exchange
Input [3]: [product#143171, plant#143170, min#144811]
Arguments: hashpartitioning(product#143171, plant#143170, 25), ENSURE_REQUIREMENTS, [plan_id=261880]
(169) ShuffleQueryStage
Output [3]: [product#143171, plant#143170, min#144811]
Arguments: 179
(170) AQEShuffleRead
Input [3]: [product#143171, plant#143170, min#144811]
Arguments: coalesced
(171) HashAggregate [codegen id : 101]
Input [3]: [product#143171, plant#143170, min#144811]
Keys [2]: [product#143171, plant#143170]
Functions [1]: [min(calculationDateTime#143087)]
Aggregate Attributes [1]: [min(calculationDateTime#143087)#143334]
Results [3]: [product#143171, plant#143170, min(calculationDateTime#143087)#143334 AS minCalculationDateTime#143327]
(172) BroadcastExchange
Input [3]: [product#143171, plant#143170, minCalculationDateTime#143327]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=263108]
(173) BroadcastQueryStage
Output [3]: [product#143171, plant#143170, minCalculationDateTime#143327]
Arguments: 182
(174) BroadcastHashJoin [codegen id : 103]
Left keys [2]: [plant#142981, product#142980]
Right keys [2]: [plant#143170, product#143171]
Join type: LeftOuter
Join condition: None
(175) Project [codegen id : 103]
Output [4]: [product#142980, plant#142981, timezonecode#143012, CASE WHEN isnotnull(minCalculationDateTime#143327) THEN minCalculationDateTime#143327 ELSE orderDateTime#142982 END AS stockDatetime#143384]
Input [7]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, product#143171, plant#143170, minCalculationDateTime#143327]
(176) Exchange
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: hashpartitioning(coalesce(plant#142981, ), isnull(plant#142981), coalesce(product#142980, ), isnull(product#142980), 25), ENSURE_REQUIREMENTS, [plan_id=263400]
(177) ShuffleQueryStage
Output [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: 183
(178) AQEShuffleRead
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: local
(179) Scan parquet
Output [3]: [product#143466, plant#143467, orderDateTime#143468]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(180) ColumnarToRow [codegen id : 25]
Input [3]: [product#143466, plant#143467, orderDateTime#143468]
(181) Exchange
Input [3]: [product#143466, plant#143467, orderDateTime#143468]
Arguments: hashpartitioning(product#143466, plant#143467, 25), REPARTITION_BY_NUM, [plan_id=226931]
(182) ShuffleQueryStage
Output [3]: [product#143466, plant#143467, orderDateTime#143468]
Arguments: 39
(183) Scan parquet
Output [8]: [internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477, systemId#143470]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>
(184) ColumnarToRow [codegen id : 26]
Input [8]: [internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477, systemId#143470]
(185) Project [codegen id : 26]
Output [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
Input [8]: [internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477, systemId#143470]
(186) Exchange
Input [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=226945]
(187) ShuffleQueryStage
Output [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
Arguments: 40
(188) Project [codegen id : 71]
Output [1]: [struct(systemId, systemId#143470, internalUUID, internalUUID#143471, plant, plant#143472, product, product#143473, blockedForReplenishmentStartingFrom, blockedForReplenishmentStartingFrom#143474, productionAspect, productionAspect#143475, salesPlant, salesPlant#143476, listing, listing#143477) AS productPlant#143478]
Input [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
(189) BroadcastExchange
Input [1]: [productPlant#143478]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID), coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID)),false), [plan_id=240579]
(190) BroadcastQueryStage
Output [1]: [productPlant#143478]
Arguments: 150
(191) BroadcastHashJoin [codegen id : 105]
Left keys [4]: [coalesce(product#143466, ), isnull(product#143466), coalesce(plant#143467, ), isnull(plant#143467)]
Right keys [4]: [coalesce(productPlant#143478.product.internalRefUUID, ), isnull(productPlant#143478.product.internalRefUUID), coalesce(productPlant#143478.plant.internalRefUUID, ), isnull(productPlant#143478.plant.internalRefUUID)]
Join type: LeftOuter
Join condition: None
(192) ReusedExchange [Reuses operator id: 25]
Output [2]: [internalUUID#143480, timezonecode#143491]
(193) BroadcastQueryStage
Output [2]: [internalUUID#143480, timezonecode#143491]
Arguments: 121
(194) BroadcastHashJoin [codegen id : 105]
Left keys [2]: [coalesce(plant#143467, ), isnull(plant#143467)]
Right keys [2]: [coalesce(internalUUID#143480, ), isnull(internalUUID#143480)]
Join type: LeftOuter
Join condition: None
(195) Project [codegen id : 105]
Output [5]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, CASE WHEN isnotnull(timezonecode#143491) THEN timezonecode#143491 ELSE UTC END AS timezonecode#143492]
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, internalUUID#143480, timezonecode#143491]
(196) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
(197) BroadcastQueryStage
Output [3]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
Arguments: 102
(198) BroadcastHashJoin [codegen id : 105]
Left keys [1]: [product#143466]
Right keys [1]: [internalUUID#143494]
Join type: LeftOuter
Join condition: None
(199) Project
Output [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, filter(unitOfMeasureConversions#143500, lambdafunction((((lambda u#143027.measurementUnit1.code = productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.code) AND (lambda u#143027.measurementUnit1.internalRefUUID = productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143498.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143498.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143505]
Input [8]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
(200) Project [codegen id : 106]
Output [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, filteredUnitOfMeasureConversions#143505.quantityDenominator AS _extract_quantityDenominator#144016, filteredUnitOfMeasureConversions#143505.quantityNumerator AS _extract_quantityNumerator#144017]
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, filteredUnitOfMeasureConversions#143505]
(201) Generate [codegen id : 106]
Input [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, _extract_quantityDenominator#144016, _extract_quantityNumerator#144017]
Arguments: explode(_extract_quantityNumerator#144017), [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, _extract_quantityDenominator#144016], true, [quantityNumerator#143506]
(202) Generate [codegen id : 106]
Input [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, _extract_quantityDenominator#144016, quantityNumerator#143506]
Arguments: explode(_extract_quantityDenominator#144016), [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, quantityNumerator#143506], true, [quantityDenominator#143507]
(203) Project [codegen id : 106]
Output [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, (cast(quantityNumerator#143506 as double) / cast(quantityDenominator#143507 as double)) AS outboundUnit#143508]
Input [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, quantityNumerator#143506, quantityDenominator#143507]
(204) Project [codegen id : 106]
Output [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, CASE WHEN (isnull(outboundUnit#143508) OR (outboundUnit#143508 <= 0.0)) THEN 1.0 ELSE outboundUnit#143508 END AS outboundUnit#143509]
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143508]
(205) ReusedExchange [Reuses operator id: 49]
Output [6]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515]
(206) BroadcastQueryStage
Output [6]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515]
Arguments: 118
(207) ReusedExchange [Reuses operator id: 62]
Output [3]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
(208) BroadcastQueryStage
Output [3]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
Arguments: 112
(209) ReusedExchange [Reuses operator id: 91]
Output [1]: [plant#143545]
(210) ShuffleQueryStage
Output [1]: [plant#143545]
Arguments: 156
(211) AQEShuffleRead
Input [1]: [plant#143545]
Arguments: coalesced
(212) HashAggregate
Input [1]: [plant#143545]
Keys [1]: [plant#143545]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143545]
(213) BroadcastHashJoin
Left keys [1]: [plantUuid#143541]
Right keys [1]: [plant#143545]
Join type: Inner
Join condition: None
(214) Project
Output [3]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
Input [4]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543, plant#143545]
(215) BroadcastHashJoin [codegen id : 87]
Left keys [2]: [coalesce(plant#143513.internalRefUUID, ), isnull(plant#143513.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#143541, ), isnull(plantUuid#143541)]
Join type: Inner
Join condition: None
(216) Generate
Input [9]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515, plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
Arguments: explode(filter(stockQuantities#143515, lambdafunction(((lambda x#143085.storageLocation.internalRefUuid = storageLocUuid#143543) AND (lambda x#143085.storageLocation.Id = storageLocId#143542)), lambda x#143085, false))), [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, plantUuid#143541], false, [stockQuantity#143588]
(217) ObjectHashAggregate
Input [7]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, plantUuid#143541, stockQuantity#143588]
Keys [5]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510]
Functions [2]: [partial_first(calculationDateTime#143514, false), partial_collect_list(stockQuantity#143588, 0, 0)]
Aggregate Attributes [3]: [first#144834, valueSet#144835, buf#144836]
Results [8]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510, first#144837, valueSet#144838, buf#144839]
(218) ObjectHashAggregate
Input [8]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510, first#144837, valueSet#144838, buf#144839]
Keys [5]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510]
Functions [2]: [first(calculationDateTime#143514, false), collect_list(stockQuantity#143588, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#143514)()#143100, collect_list(stockQuantity#143588, 0, 0)#143101]
Results [4]: [product#143512, plant#143513, first(calculationDateTime#143514)()#143100 AS calculationDateTime#143589, collect_list(stockQuantity#143588, 0, 0)#143101 AS stockQuantities#143590]
(219) Filter [codegen id : 88]
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantities#143590]
Condition : ((size(stockQuantities#143590, true) > 0) AND isnotnull(calculationDateTime#143589))
(220) Generate [codegen id : 88]
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantities#143590]
Arguments: explode(stockQuantities#143590), [product#143512, plant#143513, calculationDateTime#143589], false, [stockQuantity#143591]
(221) Filter [codegen id : 88]
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantity#143591]
Condition : (((isnotnull(stockQuantity#143591.specialStockIndicator.code) AND isnotnull(stockQuantity#143591.stockType.code)) AND (stockQuantity#143591.specialStockIndicator.code = )) AND (stockQuantity#143591.stockType.code = 01))
(222) Project [codegen id : 88]
Output [5]: [plant#143513.internalRefUUID AS plantUuid#143593, product#143512.internalRefUUID AS productUuid#143595, calculationDateTime#143589, stockQuantity#143591.storagelocation.id AS storageLocId#143596, stockQuantity#143591.quantity.measure AS quantity#143598]
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantity#143591]
(223) Exchange
Input [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: hashpartitioning(productUuid#143595, plantUuid#143593, 25), ENSURE_REQUIREMENTS, [plan_id=258135]
(224) ShuffleQueryStage
Output [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: 164
(225) AQEShuffleRead
Input [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: local
(226) BroadcastExchange
Input [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true], input[1, string, true]),false), [plan_id=260287]
(227) BroadcastQueryStage
Output [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: 176
(228) ReusedExchange [Reuses operator id: 113]
Output [3]: [product#143601, plant#143602, orderDateTime#143603]
(229) ShuffleQueryStage
Output [3]: [product#143601, plant#143602, orderDateTime#143603]
Arguments: 58
(230) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#144154, _extract_internalRefUUID#144155, _extract_internalRefUUID#144152, _extract_internalRefUUID#144153]
(231) BroadcastQueryStage
Output [4]: [_extract_code#144154, _extract_internalRefUUID#144155, _extract_internalRefUUID#144152, _extract_internalRefUUID#144153]
Arguments: 134
(232) BroadcastHashJoin [codegen id : 97]
Left keys [4]: [coalesce(product#143601, ), isnull(product#143601), coalesce(plant#143602, ), isnull(plant#143602)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144152, ), isnull(_extract_internalRefUUID#144152), coalesce(_extract_internalRefUUID#144153, ), isnull(_extract_internalRefUUID#144153)]
Join type: LeftOuter
Join condition: None
(233) Project [codegen id : 97]
Output [5]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155]
Input [7]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155, _extract_internalRefUUID#144152, _extract_internalRefUUID#144153]
(234) ReusedExchange [Reuses operator id: 78]
Output [1]: [internalUUID#143615]
(235) BroadcastQueryStage
Output [1]: [internalUUID#143615]
Arguments: 146
(236) BroadcastHashJoin [codegen id : 97]
Left keys [2]: [coalesce(plant#143602, ), isnull(plant#143602)]
Right keys [2]: [coalesce(internalUUID#143615, ), isnull(internalUUID#143615)]
Join type: LeftOuter
Join condition: None
(237) Project [codegen id : 97]
Output [5]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155]
Input [6]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155, internalUUID#143615]
(238) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
(239) BroadcastQueryStage
Output [3]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
Arguments: 104
(240) BroadcastHashJoin [codegen id : 97]
Left keys [1]: [product#143601]
Right keys [1]: [internalUUID#143629]
Join type: LeftOuter
Join condition: None
(241) Project
Output [4]: [product#143601, plant#143602, orderDateTime#143603, filter(unitOfMeasureConversions#143635, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144154) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144155)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143633.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143633.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143640]
Input [8]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155, internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
(242) Generate
Input [4]: [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640]
Arguments: explode(filteredUnitOfMeasureConversions#143640.quantityNumerator), [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640], true, [quantityNumerator#143641]
(243) Project
Output [4]: [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640.quantityDenominator AS _extract_quantityDenominator#143966]
Input [5]: [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640, quantityNumerator#143641]
(244) Generate
Input [4]: [product#143601, plant#143602, orderDateTime#143603, _extract_quantityDenominator#143966]
Arguments: explode(_extract_quantityDenominator#143966), [product#143601, plant#143602, orderDateTime#143603], true, [quantityDenominator#143642]
(245) Project
Output [3]: [product#143601, plant#143602, orderDateTime#143603]
Input [4]: [product#143601, plant#143602, orderDateTime#143603, quantityDenominator#143642]
(246) BroadcastHashJoin [codegen id : 98]
Left keys [2]: [plantUuid#143593, productUuid#143595]
Right keys [2]: [plant#143602, product#143601]
Join type: Inner
Join condition: (calculationDateTime#143589 <= orderDateTime#143603)
(247) Project [codegen id : 98]
Output [5]: [plantUuid#143593 AS plant#143645, productUuid#143595 AS product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Input [8]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598, product#143601, plant#143602, orderDateTime#143603]
(248) Exchange
Input [5]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Arguments: hashpartitioning(coalesce(plant#143645, ), isnull(plant#143645), coalesce(product#143646, ), isnull(product#143646), coalesce(storageLocId#143596, ), isnull(storageLocId#143596), coalesce(calculationDateTime#143589, 1970-01-01 00:00:00), isnull(calculationDateTime#143589), 25), ENSURE_REQUIREMENTS, [plan_id=261160]
(249) ShuffleQueryStage
Output [5]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Arguments: 178
(250) AQEShuffleRead
Input [5]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Arguments: local
(251) ReusedExchange [Reuses operator id: 162]
Output [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
(252) ShuffleQueryStage
Output [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
Arguments: 175
(253) AQEShuffleRead
Input [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
Arguments: local
(254) BroadcastExchange
Input [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true]), coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, timestamp, true], 1970-01-01 00:00:00), isnull(input[3, timestamp, true])),false), [plan_id=261902]
(255) BroadcastQueryStage
Output [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
Arguments: 180
(256) BroadcastHashJoin [codegen id : 100]
Left keys [8]: [coalesce(plant#143645, ), isnull(plant#143645), coalesce(product#143646, ), isnull(product#143646), coalesce(storageLocId#143596, ), isnull(storageLocId#143596), coalesce(calculationDateTime#143589, 1970-01-01 00:00:00), isnull(calculationDateTime#143589)]
Right keys [8]: [coalesce(plant#143782, ), isnull(plant#143782), coalesce(product#143783, ), isnull(product#143783), coalesce(storageLocId#143733, ), isnull(storageLocId#143733), coalesce(max_calc_datetime#143784, 1970-01-01 00:00:00), isnull(max_calc_datetime#143784)]
Join type: Inner
Join condition: None
(257) Project [codegen id : 100]
Output [4]: [plant#143645, product#143646, calculationDateTime#143589, quantity#143598]
Input [9]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598, plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
(258) HashAggregate [codegen id : 100]
Input [4]: [plant#143645, product#143646, calculationDateTime#143589, quantity#143598]
Keys [2]: [product#143646, plant#143645]
Functions [2]: [partial_min(calculationDateTime#143589), partial_sum(quantity#143598)]
Aggregate Attributes [2]: [min#144830, sum#144831]
Results [4]: [product#143646, plant#143645, min#144832, sum#144833]
(259) Exchange
Input [4]: [product#143646, plant#143645, min#144832, sum#144833]
Arguments: hashpartitioning(product#143646, plant#143645, 25), ENSURE_REQUIREMENTS, [plan_id=262809]
(260) ShuffleQueryStage
Output [4]: [product#143646, plant#143645, min#144832, sum#144833]
Arguments: 181
(261) AQEShuffleRead
Input [4]: [product#143646, plant#143645, min#144832, sum#144833]
Arguments: coalesced
(262) HashAggregate [codegen id : 104]
Input [4]: [product#143646, plant#143645, min#144832, sum#144833]
Keys [2]: [product#143646, plant#143645]
Functions [2]: [min(calculationDateTime#143589), sum(quantity#143598)]
Aggregate Attributes [2]: [min(calculationDateTime#143589)#143334, sum(quantity#143598)#143333]
Results [4]: [product#143646, plant#143645, min(calculationDateTime#143589)#143334 AS minCalculationDateTime#143786, round(sum(quantity#143598)#143333, 0) AS roundedQuantitySum#143787]
(263) BroadcastExchange
Input [4]: [product#143646, plant#143645, minCalculationDateTime#143786, roundedQuantitySum#143787]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=263627]
(264) BroadcastQueryStage
Output [4]: [product#143646, plant#143645, minCalculationDateTime#143786, roundedQuantitySum#143787]
Arguments: 184
(265) BroadcastHashJoin [codegen id : 106]
Left keys [2]: [plant#143467, product#143466]
Right keys [2]: [plant#143645, product#143646]
Join type: LeftOuter
Join condition: None
(266) Project [codegen id : 106]
Output [8]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, CASE WHEN isnotnull(minCalculationDateTime#143786) THEN minCalculationDateTime#143786 ELSE orderDateTime#143468 END AS stockDatetime#143788, CASE WHEN isnotnull(roundedQuantitySum#143787) THEN array(roundedQuantitySum#143787) ELSE [0.0] END AS stockBatchQuantities#143789]
Input [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, product#143646, plant#143645, minCalculationDateTime#143786, roundedQuantitySum#143787]
(267) Scan parquet
Output [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-demand-service/out/demand/10000002108/0_1_10000002108]
PushedFilters: [IsNotNull(product), IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string,demandChannel:string,demandStream:string,considerVariance:boolean,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,demandQuantity:double,demandVariance:double>>,demandPointInTimeStart:timestamp,demandPointInTimeEnd:timestamp,demandPointInTime:array<struct<demandDateTime:timestamp,demandQuantity:double,demandVariance:double>>>
(268) ColumnarToRow [codegen id : 49]
Input [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
(269) Filter [codegen id : 49]
Input [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
Condition : (isnotnull(product#143387) AND isnotnull(plant#143388))
(270) BroadcastExchange
Input [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
Arguments: HashedRelationBroadcastMode(List(input[0, string, false], input[1, string, false]),false), [plan_id=227509]
(271) BroadcastQueryStage
Output [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
Arguments: 85
(272) BroadcastHashJoin [codegen id : 106]
Left keys [2]: [product#143466, plant#143467]
Right keys [2]: [product#143387, plant#143388]
Join type: LeftOuter
Join condition: None
(273) Project [codegen id : 106]
Output [9]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, CASE WHEN isnotnull(product#143387) THEN struct(demandChannel, demandChannel#143389, demandStream, demandStream#143390, considerVariance, considerVariance#143391, demandTimeBuckets, demandTimeBuckets#143392, demandPointInTimeStart, demandPointInTimeStart#143393, demandPointInTimeEnd, demandPointInTimeEnd#143394, demandPointInTime, demandPointInTime#143395) END AS rawDemands#143412]
Input [17]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
(274) ObjectHashAggregate
Input [9]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, rawDemands#143412]
Keys [9]: [product#143466, plant#143467, orderDateTime#143468, knownfloatingpointnormalized(if (isnull(productPlant#143478)) null else named_struct(systemId, productPlant#143478.systemId, internalUUID, productPlant#143478.internalUUID, plant, productPlant#143478.plant, product, productPlant#143478.product, blockedForReplenishmentStartingFrom, productPlant#143478.blockedForReplenishmentStartingFrom, productionAspect, knownfloatingpointnormalized(if (isnull(productPlant#143478.productionAspect)) null else named_struct(productMovementPlants, productPlant#143478.productionAspect.productMovementPlants, productPlanningPlants, knownfloatingpointnormalized(if (isnull(productPlant#143478.productionAspect.productPlanningPlants)) null else named_struct(mrpType, productPlant#143478.productionAspect.productPlanningPlants.mrpType, abcIndicator, productPlant#143478.productionAspect.productPlanningPlants.abcIndicator, mrpController, productPlant#143478.productionAspect.productPlanningPlants.mrpController, minimumLotSize, knownfloatingpointnormalized(normalizenanandzero(productPlant#143478.productionAspect.productPlanningPlants.minimumLotSize)), minimumSafetyStock, knownfloatingpointnormalized(normalizenanandzero(productPlant#143478.productionAspect.productPlanningPlants.minimumSafetyStock)), logisticalRoundingProfile, productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile, sourceOfSupplyCategory, productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory)))), salesPlant, productPlant#143478.salesPlant, listing, productPlant#143478.listing)) AS productPlant#143478, timezonecode#143492, knownfloatingpointnormalized(normalizenanandzero(outboundUnit#143509)) AS outboundUnit#143509, stockDatetime#143788, knownfloatingpointnormalized(transform(stockBatchQuantities#143789, lambdafunction(knownfloatingpointnormalized(normalizenanandzero(lambda arg#150351)), lambda arg#150351, false))) AS stockBatchQuantities#143789, [7226582400000000] AS [7226582400000000]#150350]
Functions [1]: [partial_collect_list(rawDemands#143412, 0, 0)]
Aggregate Attributes [1]: [buf#144828]
Results [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000]#150350, buf#144829]
(275) ObjectHashAggregate
Input [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000]#150350, buf#144829]
Keys [9]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000]#150350]
Functions [1]: [collect_list(rawDemands#143412, 0, 0)]
Aggregate Attributes [1]: [collect_list(rawDemands#143412, 0, 0)#143431]
Results [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000] AS stockBatchShelfLifeEndDatetimes#143790, collect_list(rawDemands#143412, 0, 0)#143431 AS rawDemands#143413]
(276) DeserializeToObject [codegen id : 107]
Input [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, stockBatchShelfLifeEndDatetimes#143790, rawDemands#143413]
Arguments: createexternalrow(invoke(product#143466.toString()), invoke(plant#143467.toString()), static_invoke(DateTimeUtils.toJavaTimestamp(orderDateTime#143468)), if (isnull(productPlant#143478)) null else createexternalrow(invoke(productPlant#143478.systemId.toString()), invoke(productPlant#143478.internalUUID.toString()), if (isnull(productPlant#143478.plant)) null else createexternalrow(invoke(productPlant#143478.plant.internalRefUUID.toString()), invoke(productPlant#143478.plant.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), if (isnull(productPlant#143478.product)) null else createexternalrow(invoke(productPlant#143478.product.internalRefUUID.toString()), invoke(productPlant#143478.product.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(DateTimeUtils.toJavaDate(productPlant#143478.blockedForReplenishmentStartingFrom)), if (isnull(productPlant#143478.productionAspect)) null else createexternalrow(if (isnull(productPlant#143478.productionAspect.productMovementPlants)) null else createexternalrow(if (isnull(productPlant#143478.productionAspect.productMovementPlants.unitOfIssue)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants)) null else createexternalrow(if (isnull(productPlant#143478.productionAspect.productPlanningPlants.mrpType)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpType.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpType.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.abcIndicator)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.abcIndicator.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.abcIndicator.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.mrpController)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpController.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpController.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(java.lang.Double.valueOf(productPlant#143478.productionAspect.productPlanningPlants.minimumLotSize)), static_invoke(java.lang.Double.valueOf(productPlant#143478.productionAspect.productPlanningPlants.minimumSafetyStock)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile.logisticalRoundingProfile.toString()), StructField(internalRefUUID,StringType,true), StructField(logisticalRoundingProfile,StringType,true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(minimumLotSize,DoubleType,true), StructField(minimumSafetyStock,DoubleType,true), StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true), StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)), if (isnull(productPlant#143478.salesPlant)) null else createexternalrow(if (isnull(productPlant#143478.salesPlant.loadingGroup)) null else createexternalrow(invoke(productPlant#143478.salesPlant.loadingGroup.internalRefUUID.toString()), invoke(productPlant#143478.salesPlant.loadingGroup.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), mapobjects(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1), if (isnull(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validFrom)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validTo)), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).isListed)), StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true)), productPlant#143478.listing, Some(class scala.collection.mutable.ArraySeq)), StructField(systemId,StringType,true), StructField(internalUUID,StringType,true), StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(blockedForReplenishmentStartingFrom,DateType,true), StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true), StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)), invoke(timezonecode#143492.toString()), static_invoke(java.lang.Double.valueOf(outboundUnit#143509)), static_invoke(DateTimeUtils.toJavaTimestamp(stockDatetime#143788)), mapobjects(lambdavariable(MapObject, DoubleType, true, -2), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, DoubleType, true, -2))), stockBatchQuantities#143789, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, TimestampType, false, -3), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, TimestampType, false, -3))), stockBatchShelfLifeEndDatetimes#143790, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4), createexternalrow(invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandChannel.toString()), invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandStream.toString()), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).considerVariance)), mapobjects(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5), if (isnull(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandStartDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandVariance)), StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandTimeBuckets, Some(class scala.collection.mutable.ArraySeq)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeStart)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeEnd)), mapobjects(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6), if (isnull(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandVariance)), StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTime, Some(class scala.collection.mutable.ArraySeq)), StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)), rawDemands#143413, Some(class scala.collection.mutable.ArraySeq)), StructField(product,StringType,true), StructField(plant,StringType,true), StructField(orderDateTime,TimestampType,true), StructField(productPlant,StructType(StructField(systemId,StringType,true),StructField(internalUUID,StringType,true),StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(blockedForReplenishmentStartingFrom,DateType,true),StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true),StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)),true), StructField(timezonecode,StringType,true), StructField(outboundUnit,DoubleType,true), StructField(stockDatetime,TimestampType,true), StructField(stockBatchQuantities,ArrayType(DoubleType,true),false), StructField(stockBatchShelfLifeEndDatetimes,ArrayType(TimestampType,false),false), StructField(rawDemands,ArrayType(StructType(StructField(demandChannel,StringType,true),StructField(demandStream,StringType,true),StructField(considerVariance,BooleanType,true),StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true),StructField(demandPointInTimeStart,TimestampType,true),StructField(demandPointInTimeEnd,TimestampType,true),StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)),false),false)), obj#143452: org.apache.spark.sql.Row
(277) MapElements [codegen id : 107]
Input [1]: [obj#143452]
Arguments: com.sap.s4hana.eureka.business.crporderquantitykpiservice.core.business.datasnapshot.DemandTransformatorImpl$$Lambda/0x00000008024b0d88@2d5ca6d1, obj#143457: org.apache.spark.sql.Row
(278) SerializeFromObject [codegen id : 107]
Input [1]: [obj#143457]
Arguments: [if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(0))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, product), StringType, ObjectType(class java.lang.String)))) AS product#143462, if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(1))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, plant), StringType, ObjectType(class java.lang.String)))) AS plant#143463]
(279) Exchange
Input [2]: [product#143462, plant#143463]
Arguments: hashpartitioning(coalesce(plant#143463, ), isnull(plant#143463), coalesce(product#143462, ), isnull(product#143462), 25), ENSURE_REQUIREMENTS, [plan_id=263848]
(280) ShuffleQueryStage
Output [2]: [product#143462, plant#143463]
Arguments: 185
(281) AQEShuffleRead
Input [2]: [product#143462, plant#143463]
Arguments: local
(282) BroadcastExchange
Input [2]: [product#143462, plant#143463]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=264019]
(283) BroadcastQueryStage
Output [2]: [product#143462, plant#143463]
Arguments: 186
(284) BroadcastHashJoin [codegen id : 108]
Left keys [4]: [coalesce(plant#142981, ), isnull(plant#142981), coalesce(product#142980, ), isnull(product#142980)]
Right keys [4]: [coalesce(plant#143463, ), isnull(plant#143463), coalesce(product#143462, ), isnull(product#143462)]
Join type: LeftOuter
Join condition: None
(285) Project [codegen id : 108]
Output [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Input [6]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384, product#143462, plant#143463]
(286) Exchange
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: hashpartitioning(plant#142981, 25), ENSURE_REQUIREMENTS, [plan_id=264090]
(287) ShuffleQueryStage
Output [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: 187
(288) AQEShuffleRead
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: local
(289) BroadcastExchange
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true]),false), [plan_id=264140]
(290) BroadcastQueryStage
Output [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: 188
(291) ReusedExchange [Reuses operator id: 55]
Output [1]: [distributionCenters#143861]
(292) ShuffleQueryStage
Output [1]: [distributionCenters#143861]
Arguments: 87
(293) Generate [codegen id : 60]
Input [1]: [distributionCenters#143861]
Arguments: explode(distributionCenters#143861), false, [distributionCenters#143877]
(294) Filter [codegen id : 60]
Input [1]: [distributionCenters#143877]
Condition : (((size(distributionCenters#143877.storageLocations.internalRefUUID, true) > 0) AND isnotnull(distributionCenters#143877.storageLocations.internalRefUUID)) AND isnotnull(distributionCenters#143877.internalRefUUID))
(295) Project [codegen id : 60]
Output [2]: [distributionCenters#143877.internalRefUUID AS _extract_internalRefUUID#143980, distributionCenters#143877.storageLocations.internalRefUUID AS _extract_internalRefUUID#144165]
Input [1]: [distributionCenters#143877]
(296) Generate [codegen id : 60]
Input [2]: [_extract_internalRefUUID#143980, _extract_internalRefUUID#144165]
Arguments: explode(_extract_internalRefUUID#144165), [_extract_internalRefUUID#143980], false, [storageLocations#143878]
(297) Filter [codegen id : 60]
Input [2]: [_extract_internalRefUUID#143980, storageLocations#143878]
Condition : isnotnull(storageLocations#143878)
(298) Project [codegen id : 60]
Output [2]: [_extract_internalRefUUID#143980 AS plantUuid#143880, storageLocations#143878 AS storageLocUuid#143882]
Input [2]: [_extract_internalRefUUID#143980, storageLocations#143878]
(299) BroadcastExchange
Input [2]: [plantUuid#143880, storageLocUuid#143882]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=231566]
(300) BroadcastQueryStage
Output [2]: [plantUuid#143880, storageLocUuid#143882]
Arguments: 115
(301) ReusedExchange [Reuses operator id: 91]
Output [1]: [plant#143884]
(302) ShuffleQueryStage
Output [1]: [plant#143884]
Arguments: 160
(303) HashAggregate
Input [1]: [plant#143884]
Keys [1]: [plant#143884]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143884]
(304) BroadcastHashJoin
Left keys [1]: [plantUuid#143880]
Right keys [1]: [plant#143884]
Join type: Inner
Join condition: None
(305) Project
Output [2]: [plantUuid#143880 AS plant#143854, storageLocUuid#143882]
Input [3]: [plantUuid#143880, storageLocUuid#143882, plant#143884]
(306) BroadcastHashJoin [codegen id : 109]
Left keys [1]: [plant#142981]
Right keys [1]: [plant#143854]
Join type: Inner
Join condition: None
(307) Project [codegen id : 109]
Output [5]: [plant#142981, product#142980, timezonecode#143012, stockDatetime#143384, storageLocUuid#143882]
Input [6]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384, plant#143854, storageLocUuid#143882]
(308) Scan parquet
Output [6]: [internalUUID#143816, id#143817, supplier#143819, isMarkedForDeletion#143822, items#143823, systemId#143815]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-procurement-purchaseorder]
PushedFilters: [IsNotNull(isMarkedForDeletion), Not(EqualTo(isMarkedForDeletion,true)), IsNotNull(items)]
ReadSchema: struct<internalUUID:string,id:string,supplier:struct<internalRefUUID:string,id:string>,isMarkedForDeletion:boolean,items:array<struct<number:string,completeIndicator:boolean,returnIndicator:boolean,deletionIndicator:boolean,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:double,quantityUnit:struct<internalRefUUID:string,code:string>,quantityNumerator:double,quantityDenominator:double,scheduleLines:array<struct<number:string,deliveryAt:timestamp,quantity:double,quantityUnit:struct<internalRefUUID:string,code:string>,receivedQuantity:double,deliveredQuantity:double,productAvailabilityDateTime:timestamp>>,confirmations:array<struct<number:string,confirmationCategory:struct<internalRefUUID:string,code:string>,confirmationDeliveryDate:date,confirmedDeliveryAt:timestamp,confirmedQuantity:double,mrpRelevantQuantity:double,confirmationCreationCategory:string,deletionIndicator:boolean,isRelevantForConfirmation:boolean,delivery:struct<internalRefUUID:string,id:string>,deliveryItemNumber:string>>>>>
(309) ColumnarToRow [codegen id : 55]
Input [6]: [internalUUID#143816, id#143817, supplier#143819, isMarkedForDeletion#143822, items#143823, systemId#143815]
(310) Filter [codegen id : 55]
Input [6]: [internalUUID#143816, id#143817, supplier#143819, isMarkedForDeletion#143822, items#143823, systemId#143815]
Condition : (((isnotnull(isMarkedForDeletion#143822) AND NOT isMarkedForDeletion#143822) AND (size(items#143823, true) > 0)) AND isnotnull(items#143823))
(311) Project [codegen id : 55]
Output [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Input [6]: [internalUUID#143816, id#143817, supplier#143819, isMarkedForDeletion#143822, items#143823, systemId#143815]
(312) Exchange
Input [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=227742]
(313) ShuffleQueryStage
Output [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Arguments: 96
(314) Generate [codegen id : 72]
Input [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Arguments: explode(items#143823), [internalUUID#143816, id#143817, supplier#143819], false, [item#143827]
(315) Filter [codegen id : 72]
Input [4]: [internalUUID#143816, id#143817, supplier#143819, item#143827]
Condition : ((((isnotnull(item#143827.deletionIndicator) AND isnotnull(item#143827.completeIndicator)) AND isnotnull(item#143827.returnIndicator)) AND ((NOT item#143827.deletionIndicator AND NOT item#143827.completeIndicator) AND NOT item#143827.returnIndicator)) AND ((isnotnull(item#143827.product.internalRefUUID) AND isnotnull(item#143827.plant.internalRefUUID)) AND isnotnull(item#143827.storageLocation.internalRefUUID)))
(316) Project [codegen id : 72]
Output [14]: [id#143817, internalUUID#143816, supplier#143819.internalRefUUID AS supplierUuid#143824, item#143827.plant.internalRefUUID AS plant#143832, item#143827.product.internalRefUUID AS product#143833, item#143827.plant.id AS externalPlantId#143834, item#143827.product.id AS externalProductId#143835, item#143827.storageLocation.internalRefUUID AS storageLocUuid#143836, item#143827.quantityUnit.code AS quantityUom#143837, item#143827.quantityUnit.internalRefUUID AS quantityUomUuid#143838, item#143827.quantityNumerator AS quantityNumerator#143839, item#143827.quantityDenominator AS quantityDenominator#143840, item#143827.scheduleLines AS scheduleLines#143841, item#143827.confirmations AS confirmations#143842]
Input [4]: [internalUUID#143816, id#143817, supplier#143819, item#143827]
(317) BroadcastExchange
Input [14]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842]
Arguments: HashedRelationBroadcastMode(List(input[4, string, true], input[3, string, true], input[7, string, true]),false), [plan_id=241931]
(318) BroadcastQueryStage
Output [14]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842]
Arguments: 151
(319) BroadcastHashJoin [codegen id : 109]
Left keys [3]: [product#142980, plant#142981, storageLocUuid#143882]
Right keys [3]: [product#143833, plant#143832, storageLocUuid#143836]
Join type: Inner
Join condition: None
(320) Project
Output [17]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842, stockDatetime#143384, timezonecode#143012, filter(confirmations#143842, lambdafunction(((NOT lambda x_29#143933.deletionIndicator AND lambda x_29#143933.isRelevantForConfirmation) AND isnotnull(lambda x_29#143933.confirmedDeliveryAt)), lambda x_29#143933, false)) AS validConfirmations#143932]
Input [19]: [plant#142981, product#142980, timezonecode#143012, stockDatetime#143384, storageLocUuid#143882, id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842]
(321) ResultQueryStage
Output [17]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842, stockDatetime#143384, timezonecode#143012, validConfirmations#143932]
Arguments: 189
(322) Filter
Input [3]: [product#142980, plant#142981, orderDateTime#142982]
Condition : (isnotnull(plant#142981) AND isnotnull(product#142980))
(323) Exchange
Input [3]: [product#142980, plant#142981, orderDateTime#142982]
Arguments: hashpartitioning(product#142980, plant#142981, 25), REPARTITION_BY_NUM, [plan_id=216939]
(324) Project
Output [4]: [plant#142987.internalRefUUID AS _extract_internalRefUUID#144131, product#142988.internalRefUUID AS _extract_internalRefUUID#144130, productionAspect#142990.productMovementPlants.unitOfIssue.code AS _extract_code#144128, productionAspect#142990.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144129]
Input [4]: [plant#142987, product#142988, productionAspect#142990, systemId#142985]
(325) Exchange
Input [4]: [_extract_internalRefUUID#144131, _extract_internalRefUUID#144130, _extract_code#144128, _extract_internalRefUUID#144129]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216510]
(326) Project
Output [4]: [_extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
Input [4]: [_extract_internalRefUUID#144131, _extract_internalRefUUID#144130, _extract_code#144128, _extract_internalRefUUID#144129]
(327) BroadcastExchange
Input [4]: [_extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217138]
(328) BroadcastHashJoin
Left keys [4]: [coalesce(product#142980, ), isnull(product#142980), coalesce(plant#142981, ), isnull(plant#142981)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144130, ), isnull(_extract_internalRefUUID#144130), coalesce(_extract_internalRefUUID#144131, ), isnull(_extract_internalRefUUID#144131)]
Join type: LeftOuter
Join condition: None
(329) Project
Output [5]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129]
Input [7]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, _extract_internalRefUUID#144130, _extract_internalRefUUID#144131]
(330) Project
Output [2]: [internalUUID#142998, address#143004.timeZone.code AS timezonecode#143009]
Input [3]: [internalUUID#142998, address#143004, systemId#142997]
(331) Exchange
Input [2]: [internalUUID#142998, timezonecode#143009]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216516]
(332) BroadcastExchange
Input [2]: [internalUUID#142998, timezonecode#143009]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217142]
(333) BroadcastHashJoin
Left keys [2]: [coalesce(plant#142981, ), isnull(plant#142981)]
Right keys [2]: [coalesce(internalUUID#142998, ), isnull(internalUUID#142998)]
Join type: LeftOuter
Join condition: None
(334) Project
Output [6]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, CASE WHEN isnotnull(timezonecode#143009) THEN timezonecode#143009 ELSE UTC END AS timezonecode#143012]
Input [7]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, internalUUID#142998, timezonecode#143009]
(335) Filter
Input [4]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020, systemId#143013]
Condition : isnotnull(internalUUID#143014)
(336) Project
Output [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Input [4]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020, systemId#143013]
(337) Exchange
Input [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216522]
(338) BroadcastExchange
Input [3]: [internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217146]
(339) BroadcastHashJoin
Left keys [1]: [product#142980]
Right keys [1]: [internalUUID#143014]
Join type: LeftOuter
Join condition: None
(340) Project
Output [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filter(unitOfMeasureConversions#143020, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144128) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144129)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143018.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143018.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143026]
Input [9]: [product#142980, plant#142981, orderDateTime#142982, _extract_code#144128, _extract_internalRefUUID#144129, timezonecode#143012, internalUUID#143014, baseUnitOfMeasure#143018, unitOfMeasureConversions#143020]
(341) Generate
Input [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026]
Arguments: explode(filteredUnitOfMeasureConversions#143026.quantityNumerator), [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026], true, [quantityNumerator#143034]
(342) Project
Output [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026.quantityDenominator AS _extract_quantityDenominator#143934]
Input [6]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, filteredUnitOfMeasureConversions#143026, quantityNumerator#143034]
(343) Generate
Input [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, _extract_quantityDenominator#143934]
Arguments: explode(_extract_quantityDenominator#143934), [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012], true, [quantityDenominator#143037]
(344) Project
Output [4]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012]
Input [5]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, quantityDenominator#143037]
(345) Sort
Input [4]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012]
Arguments: [product#142980 ASC NULLS FIRST, plant#142981 ASC NULLS FIRST], false, 0
(346) Filter
Input [6]: [internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, systemId#143076]
Condition : (isnotnull(plant#143079.internalRefUUID) AND isnotnull(product#143078.internalRefUUID))
(347) Project
Output [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Input [6]: [internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, systemId#143076]
(348) Exchange
Input [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216532]
(349) BroadcastExchange
Input [6]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=217177]
(350) Filter
Input [2]: [marketUnit#143040, distributionCenters#143046]
Condition : (((marketUnit#143040 <=> Daily_grocery) AND (size(distributionCenters#143046, true) > 0)) AND isnotnull(distributionCenters#143046))
(351) Project
Output [1]: [distributionCenters#143046]
Input [2]: [marketUnit#143040, distributionCenters#143046]
(352) Exchange
Input [1]: [distributionCenters#143046]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216536]
(353) Generate
Input [1]: [distributionCenters#143046]
Arguments: explode(distributionCenters#143046), false, [distributionCenters#143064]
(354) Filter
Input [1]: [distributionCenters#143064]
Condition : (((size(distributionCenters#143064.storageLocations, true) > 0) AND isnotnull(distributionCenters#143064.storageLocations)) AND isnotnull(distributionCenters#143064.internalRefUUID))
(355) Project
Output [2]: [distributionCenters#143064.internalRefUUID AS _extract_internalRefUUID#143938, distributionCenters#143064.storageLocations AS _extract_storageLocations#143939]
Input [1]: [distributionCenters#143064]
(356) Generate
Input [2]: [_extract_internalRefUUID#143938, _extract_storageLocations#143939]
Arguments: explode(_extract_storageLocations#143939), [_extract_internalRefUUID#143938], false, [storageLocations#143071]
(357) Project
Output [3]: [_extract_internalRefUUID#143938 AS plantUuid#143066, storageLocations#143071.id AS storageLocId#143072, storageLocations#143071.internalRefUUID AS storageLocUuid#143073]
Input [2]: [_extract_internalRefUUID#143938, storageLocations#143071]
(358) BroadcastExchange
Input [3]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217173]
(359) Filter
Input [2]: [product#143336, plant#143337]
Condition : isnotnull(plant#143337)
(360) Exchange
Input [2]: [product#143336, plant#143337]
Arguments: hashpartitioning(product#143336, plant#143337, 25), REPARTITION_BY_NUM, [plan_id=216950]
(361) Scan parquet
Output [4]: [plant#143342, product#143343, productionAspect#143345, systemId#143340]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(362) Project
Output [4]: [plant#143342.internalRefUUID AS _extract_internalRefUUID#144133, product#143343.internalRefUUID AS _extract_internalRefUUID#144132, productionAspect#143345.productMovementPlants.unitOfIssue.code AS _extract_code#144134, productionAspect#143345.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144135]
Input [4]: [plant#143342, product#143343, productionAspect#143345, systemId#143340]
(363) Exchange
Input [4]: [_extract_internalRefUUID#144133, _extract_internalRefUUID#144132, _extract_code#144134, _extract_internalRefUUID#144135]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216548]
(364) Project
Output [4]: [_extract_code#144134, _extract_internalRefUUID#144135, _extract_internalRefUUID#144132, _extract_internalRefUUID#144133]
Input [4]: [_extract_internalRefUUID#144133, _extract_internalRefUUID#144132, _extract_code#144134, _extract_internalRefUUID#144135]
(365) BroadcastExchange
Input [4]: [_extract_code#144134, _extract_internalRefUUID#144135, _extract_internalRefUUID#144132, _extract_internalRefUUID#144133]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217153]
(366) BroadcastHashJoin
Left keys [4]: [coalesce(product#143336, ), isnull(product#143336), coalesce(plant#143337, ), isnull(plant#143337)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144132, ), isnull(_extract_internalRefUUID#144132), coalesce(_extract_internalRefUUID#144133, ), isnull(_extract_internalRefUUID#144133)]
Join type: LeftOuter
Join condition: None
(367) Project
Output [4]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135]
Input [6]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135, _extract_internalRefUUID#144132, _extract_internalRefUUID#144133]
(368) Project
Output [1]: [internalUUID#143350]
Input [2]: [internalUUID#143350, systemId#143349]
(369) Exchange
Input [1]: [internalUUID#143350]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216554]
(370) BroadcastExchange
Input [1]: [internalUUID#143350]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217157]
(371) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143337, ), isnull(plant#143337)]
Right keys [2]: [coalesce(internalUUID#143350, ), isnull(internalUUID#143350)]
Join type: LeftOuter
Join condition: None
(372) Project
Output [4]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135]
Input [5]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135, internalUUID#143350]
(373) Scan parquet
Output [4]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370, systemId#143363]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(374) Filter
Input [4]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370, systemId#143363]
Condition : isnotnull(internalUUID#143364)
(375) Project
Output [3]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
Input [4]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370, systemId#143363]
(376) Exchange
Input [3]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216560]
(377) BroadcastExchange
Input [3]: [internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217161]
(378) BroadcastHashJoin
Left keys [1]: [product#143336]
Right keys [1]: [internalUUID#143364]
Join type: LeftOuter
Join condition: None
(379) Project
Output [2]: [plant#143337, filter(unitOfMeasureConversions#143370, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144134) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144135)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143368.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143368.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143375]
Input [7]: [product#143336, plant#143337, _extract_code#144134, _extract_internalRefUUID#144135, internalUUID#143364, baseUnitOfMeasure#143368, unitOfMeasureConversions#143370]
(380) Generate
Input [2]: [plant#143337, filteredUnitOfMeasureConversions#143375]
Arguments: explode(filteredUnitOfMeasureConversions#143375.quantityNumerator), [plant#143337, filteredUnitOfMeasureConversions#143375], true, [quantityNumerator#143376]
(381) Project
Output [2]: [plant#143337, filteredUnitOfMeasureConversions#143375.quantityDenominator AS _extract_quantityDenominator#143940]
Input [3]: [plant#143337, filteredUnitOfMeasureConversions#143375, quantityNumerator#143376]
(382) Generate
Input [2]: [plant#143337, _extract_quantityDenominator#143940]
Arguments: explode(_extract_quantityDenominator#143940), [plant#143337], true, [quantityDenominator#143377]
(383) Project
Output [1]: [plant#143337]
Input [2]: [plant#143337, quantityDenominator#143377]
(384) HashAggregate
Input [1]: [plant#143337]
Keys [1]: [plant#143337]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143337]
(385) Exchange
Input [1]: [plant#143337]
Arguments: hashpartitioning(plant#143337, 25), ENSURE_REQUIREMENTS, [plan_id=217170]
(386) HashAggregate
Input [1]: [plant#143337]
Keys [1]: [plant#143337]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143337]
(387) BroadcastHashJoin
Left keys [1]: [plantUuid#143066]
Right keys [1]: [plant#143337]
Join type: Inner
Join condition: None
(388) Project
Output [3]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Input [4]: [plantUuid#143066, storageLocId#143072, storageLocUuid#143073, plant#143337]
(389) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143079.internalRefUUID, ), isnull(plant#143079.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#143066, ), isnull(plantUuid#143066)]
Join type: Inner
Join condition: None
(390) Generate
Input [9]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, stockQuantities#143081, plantUuid#143066, storageLocId#143072, storageLocUuid#143073]
Arguments: explode(filter(stockQuantities#143081, lambdafunction(((lambda x#143085.storageLocation.internalRefUuid = storageLocUuid#143073) AND (lambda x#143085.storageLocation.Id = storageLocId#143072)), lambda x#143085, false))), [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, plantUuid#143066], false, [stockQuantity#143086]
(391) ObjectHashAggregate
Input [7]: [systemId#143076, internalUUID#143077, product#143078, plant#143079, calculationDateTime#143080, plantUuid#143066, stockQuantity#143086]
Keys [5]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076]
Functions [2]: [partial_first(calculationDateTime#143080, false), partial_collect_list(stockQuantity#143086, 0, 0)]
Aggregate Attributes [3]: [first#144812, valueSet#144813, buf#144814]
Results [8]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076, first#144815, valueSet#144816, buf#144817]
(392) ObjectHashAggregate
Input [8]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076, first#144815, valueSet#144816, buf#144817]
Keys [5]: [plantUuid#143066, internalUUID#143077, plant#143079, product#143078, systemId#143076]
Functions [2]: [first(calculationDateTime#143080, false), collect_list(stockQuantity#143086, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#143080)()#143100, collect_list(stockQuantity#143086, 0, 0)#143101]
Results [4]: [product#143078, plant#143079, first(calculationDateTime#143080)()#143100 AS calculationDateTime#143087, collect_list(stockQuantity#143086, 0, 0)#143101 AS stockQuantities#143088]
(393) Filter
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantities#143088]
Condition : ((size(stockQuantities#143088, true) > 0) AND isnotnull(calculationDateTime#143087))
(394) Generate
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantities#143088]
Arguments: explode(stockQuantities#143088), [product#143078, plant#143079, calculationDateTime#143087], false, [stockQuantity#143103]
(395) Filter
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantity#143103]
Condition : (((isnotnull(stockQuantity#143103.specialStockIndicator.code) AND isnotnull(stockQuantity#143103.stockType.code)) AND (stockQuantity#143103.specialStockIndicator.code = )) AND (stockQuantity#143103.stockType.code = 01))
(396) Project
Output [4]: [plant#143079.internalRefUUID AS plantUuid#143105, product#143078.internalRefUUID AS productUuid#143107, calculationDateTime#143087, stockQuantity#143103.storagelocation.id AS storageLocId#143108]
Input [4]: [product#143078, plant#143079, calculationDateTime#143087, stockQuantity#143103]
(397) Exchange
Input [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: hashpartitioning(productUuid#143107, plantUuid#143105, 25), ENSURE_REQUIREMENTS, [plan_id=217203]
(398) Sort
Input [4]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108]
Arguments: [productUuid#143107 ASC NULLS FIRST, plantUuid#143105 ASC NULLS FIRST], false, 0
(399) Filter
Input [3]: [product#143122, plant#143123, orderDateTime#143124]
Condition : ((isnotnull(orderDateTime#143124) AND isnotnull(plant#143123)) AND isnotnull(product#143122))
(400) Exchange
Input [3]: [product#143122, plant#143123, orderDateTime#143124]
Arguments: hashpartitioning(product#143122, plant#143123, 25), REPARTITION_BY_NUM, [plan_id=216973]
(401) Scan parquet
Output [4]: [plant#143128, product#143129, productionAspect#143131, systemId#143126]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(402) Project
Output [4]: [plant#143128.internalRefUUID AS _extract_internalRefUUID#144136, product#143129.internalRefUUID AS _extract_internalRefUUID#144137, productionAspect#143131.productMovementPlants.unitOfIssue.code AS _extract_code#144138, productionAspect#143131.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144139]
Input [4]: [plant#143128, product#143129, productionAspect#143131, systemId#143126]
(403) Exchange
Input [4]: [_extract_internalRefUUID#144136, _extract_internalRefUUID#144137, _extract_code#144138, _extract_internalRefUUID#144139]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216585]
(404) Project
Output [4]: [_extract_code#144138, _extract_internalRefUUID#144139, _extract_internalRefUUID#144137, _extract_internalRefUUID#144136]
Input [4]: [_extract_internalRefUUID#144136, _extract_internalRefUUID#144137, _extract_code#144138, _extract_internalRefUUID#144139]
(405) BroadcastExchange
Input [4]: [_extract_code#144138, _extract_internalRefUUID#144139, _extract_internalRefUUID#144137, _extract_internalRefUUID#144136]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217186]
(406) BroadcastHashJoin
Left keys [4]: [coalesce(product#143122, ), isnull(product#143122), coalesce(plant#143123, ), isnull(plant#143123)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144137, ), isnull(_extract_internalRefUUID#144137), coalesce(_extract_internalRefUUID#144136, ), isnull(_extract_internalRefUUID#144136)]
Join type: LeftOuter
Join condition: None
(407) Project
Output [5]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139]
Input [7]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139, _extract_internalRefUUID#144137, _extract_internalRefUUID#144136]
(408) Scan parquet
Output [2]: [internalUUID#143136, systemId#143135]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(409) Project
Output [1]: [internalUUID#143136]
Input [2]: [internalUUID#143136, systemId#143135]
(410) Exchange
Input [1]: [internalUUID#143136]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216591]
(411) BroadcastExchange
Input [1]: [internalUUID#143136]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217190]
(412) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143123, ), isnull(plant#143123)]
Right keys [2]: [coalesce(internalUUID#143136, ), isnull(internalUUID#143136)]
Join type: LeftOuter
Join condition: None
(413) Project
Output [5]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139]
Input [6]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139, internalUUID#143136]
(414) Scan parquet
Output [4]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156, systemId#143149]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(415) Filter
Input [4]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156, systemId#143149]
Condition : isnotnull(internalUUID#143150)
(416) Project
Output [3]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
Input [4]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156, systemId#143149]
(417) Exchange
Input [3]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216597]
(418) BroadcastExchange
Input [3]: [internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217194]
(419) BroadcastHashJoin
Left keys [1]: [product#143122]
Right keys [1]: [internalUUID#143150]
Join type: LeftOuter
Join condition: None
(420) Project
Output [4]: [product#143122, plant#143123, orderDateTime#143124, filter(unitOfMeasureConversions#143156, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144138) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144139)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143154.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143154.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143161]
Input [8]: [product#143122, plant#143123, orderDateTime#143124, _extract_code#144138, _extract_internalRefUUID#144139, internalUUID#143150, baseUnitOfMeasure#143154, unitOfMeasureConversions#143156]
(421) Generate
Input [4]: [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161]
Arguments: explode(filteredUnitOfMeasureConversions#143161.quantityNumerator), [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161], true, [quantityNumerator#143162]
(422) Project
Output [4]: [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161.quantityDenominator AS _extract_quantityDenominator#143944]
Input [5]: [product#143122, plant#143123, orderDateTime#143124, filteredUnitOfMeasureConversions#143161, quantityNumerator#143162]
(423) Generate
Input [4]: [product#143122, plant#143123, orderDateTime#143124, _extract_quantityDenominator#143944]
Arguments: explode(_extract_quantityDenominator#143944), [product#143122, plant#143123, orderDateTime#143124], true, [quantityDenominator#143163]
(424) Project
Output [3]: [product#143122, plant#143123, orderDateTime#143124]
Input [4]: [product#143122, plant#143123, orderDateTime#143124, quantityDenominator#143163]
(425) Sort
Input [3]: [product#143122, plant#143123, orderDateTime#143124]
Arguments: [product#143122 ASC NULLS FIRST, plant#143123 ASC NULLS FIRST], false, 0
(426) SortMergeJoin
Left keys [2]: [productUuid#143107, plantUuid#143105]
Right keys [2]: [product#143122, plant#143123]
Join type: Inner
Join condition: (calculationDateTime#143087 <= orderDateTime#143124)
(427) Project
Output [4]: [plantUuid#143105 AS plant#143170, productUuid#143107 AS product#143171, storageLocId#143108, calculationDateTime#143087]
Input [7]: [plantUuid#143105, productUuid#143107, calculationDateTime#143087, storageLocId#143108, product#143122, plant#143123, orderDateTime#143124]
(428) Exchange
Input [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: hashpartitioning(coalesce(plant#143170, ), isnull(plant#143170), coalesce(product#143171, ), isnull(product#143171), coalesce(storageLocId#143108, ), isnull(storageLocId#143108), coalesce(calculationDateTime#143087, 1970-01-01 00:00:00), isnull(calculationDateTime#143087), 25), ENSURE_REQUIREMENTS, [plan_id=217267]
(429) Sort
Input [4]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087]
Arguments: [coalesce(plant#143170, ) ASC NULLS FIRST, isnull(plant#143170) ASC NULLS FIRST, coalesce(product#143171, ) ASC NULLS FIRST, isnull(product#143171) ASC NULLS FIRST, coalesce(storageLocId#143108, ) ASC NULLS FIRST, isnull(storageLocId#143108) ASC NULLS FIRST, coalesce(calculationDateTime#143087, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(calculationDateTime#143087) ASC NULLS FIRST], false, 0
(430) Scan parquet
Output [6]: [internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184, systemId#143179]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(431) Filter
Input [6]: [internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184, systemId#143179]
Condition : (isnotnull(plant#143182.internalRefUUID) AND isnotnull(product#143181.internalRefUUID))
(432) Project
Output [6]: [systemId#143179, internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184]
Input [6]: [internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184, systemId#143179]
(433) Exchange
Input [6]: [systemId#143179, internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216609]
(434) BroadcastExchange
Input [6]: [systemId#143179, internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=217232]
(435) Scan parquet
Output [2]: [marketUnit#143185, distributionCenters#143191]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(436) Filter
Input [2]: [marketUnit#143185, distributionCenters#143191]
Condition : (((marketUnit#143185 <=> Daily_grocery) AND (size(distributionCenters#143191, true) > 0)) AND isnotnull(distributionCenters#143191))
(437) Project
Output [1]: [distributionCenters#143191]
Input [2]: [marketUnit#143185, distributionCenters#143191]
(438) Exchange
Input [1]: [distributionCenters#143191]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216613]
(439) Generate
Input [1]: [distributionCenters#143191]
Arguments: explode(distributionCenters#143191), false, [distributionCenters#143207]
(440) Filter
Input [1]: [distributionCenters#143207]
Condition : (((size(distributionCenters#143207.storageLocations, true) > 0) AND isnotnull(distributionCenters#143207.storageLocations)) AND isnotnull(distributionCenters#143207.internalRefUUID))
(441) Project
Output [2]: [distributionCenters#143207.internalRefUUID AS _extract_internalRefUUID#143948, distributionCenters#143207.storageLocations AS _extract_storageLocations#143949]
Input [1]: [distributionCenters#143207]
(442) Generate
Input [2]: [_extract_internalRefUUID#143948, _extract_storageLocations#143949]
Arguments: explode(_extract_storageLocations#143949), [_extract_internalRefUUID#143948], false, [storageLocations#143208]
(443) Project
Output [3]: [_extract_internalRefUUID#143948 AS plantUuid#143210, storageLocations#143208.id AS storageLocId#143211, storageLocations#143208.internalRefUUID AS storageLocUuid#143212]
Input [2]: [_extract_internalRefUUID#143948, storageLocations#143208]
(444) BroadcastExchange
Input [3]: [plantUuid#143210, storageLocId#143211, storageLocUuid#143212]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217228]
(445) Scan parquet
Output [2]: [product#143213, plant#143214]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(446) Filter
Input [2]: [product#143213, plant#143214]
Condition : isnotnull(plant#143214)
(447) Exchange
Input [2]: [product#143213, plant#143214]
Arguments: hashpartitioning(product#143213, plant#143214, 25), REPARTITION_BY_NUM, [plan_id=216986]
(448) Scan parquet
Output [4]: [plant#143219, product#143220, productionAspect#143222, systemId#143217]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(449) Project
Output [4]: [plant#143219.internalRefUUID AS _extract_internalRefUUID#144143, product#143220.internalRefUUID AS _extract_internalRefUUID#144140, productionAspect#143222.productMovementPlants.unitOfIssue.code AS _extract_code#144141, productionAspect#143222.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144142]
Input [4]: [plant#143219, product#143220, productionAspect#143222, systemId#143217]
(450) Exchange
Input [4]: [_extract_internalRefUUID#144143, _extract_internalRefUUID#144140, _extract_code#144141, _extract_internalRefUUID#144142]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216625]
(451) Project
Output [4]: [_extract_code#144141, _extract_internalRefUUID#144142, _extract_internalRefUUID#144140, _extract_internalRefUUID#144143]
Input [4]: [_extract_internalRefUUID#144143, _extract_internalRefUUID#144140, _extract_code#144141, _extract_internalRefUUID#144142]
(452) BroadcastExchange
Input [4]: [_extract_code#144141, _extract_internalRefUUID#144142, _extract_internalRefUUID#144140, _extract_internalRefUUID#144143]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217208]
(453) BroadcastHashJoin
Left keys [4]: [coalesce(product#143213, ), isnull(product#143213), coalesce(plant#143214, ), isnull(plant#143214)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144140, ), isnull(_extract_internalRefUUID#144140), coalesce(_extract_internalRefUUID#144143, ), isnull(_extract_internalRefUUID#144143)]
Join type: LeftOuter
Join condition: None
(454) Project
Output [4]: [product#143213, plant#143214, _extract_code#144141, _extract_internalRefUUID#144142]
Input [6]: [product#143213, plant#143214, _extract_code#144141, _extract_internalRefUUID#144142, _extract_internalRefUUID#144140, _extract_internalRefUUID#144143]
(455) Scan parquet
Output [2]: [internalUUID#143227, systemId#143226]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(456) Project
Output [1]: [internalUUID#143227]
Input [2]: [internalUUID#143227, systemId#143226]
(457) Exchange
Input [1]: [internalUUID#143227]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216631]
(458) BroadcastExchange
Input [1]: [internalUUID#143227]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217212]
(459) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143214, ), isnull(plant#143214)]
Right keys [2]: [coalesce(internalUUID#143227, ), isnull(internalUUID#143227)]
Join type: LeftOuter
Join condition: None
(460) Project
Output [4]: [product#143213, plant#143214, _extract_code#144141, _extract_internalRefUUID#144142]
Input [5]: [product#143213, plant#143214, _extract_code#144141, _extract_internalRefUUID#144142, internalUUID#143227]
(461) Scan parquet
Output [4]: [internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247, systemId#143240]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(462) Filter
Input [4]: [internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247, systemId#143240]
Condition : isnotnull(internalUUID#143241)
(463) Project
Output [3]: [internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247]
Input [4]: [internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247, systemId#143240]
(464) Exchange
Input [3]: [internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216637]
(465) BroadcastExchange
Input [3]: [internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217216]
(466) BroadcastHashJoin
Left keys [1]: [product#143213]
Right keys [1]: [internalUUID#143241]
Join type: LeftOuter
Join condition: None
(467) Project
Output [2]: [plant#143214, filter(unitOfMeasureConversions#143247, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144141) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144142)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143245.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143245.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143252]
Input [7]: [product#143213, plant#143214, _extract_code#144141, _extract_internalRefUUID#144142, internalUUID#143241, baseUnitOfMeasure#143245, unitOfMeasureConversions#143247]
(468) Generate
Input [2]: [plant#143214, filteredUnitOfMeasureConversions#143252]
Arguments: explode(filteredUnitOfMeasureConversions#143252.quantityNumerator), [plant#143214, filteredUnitOfMeasureConversions#143252], true, [quantityNumerator#143253]
(469) Project
Output [2]: [plant#143214, filteredUnitOfMeasureConversions#143252.quantityDenominator AS _extract_quantityDenominator#143950]
Input [3]: [plant#143214, filteredUnitOfMeasureConversions#143252, quantityNumerator#143253]
(470) Generate
Input [2]: [plant#143214, _extract_quantityDenominator#143950]
Arguments: explode(_extract_quantityDenominator#143950), [plant#143214], true, [quantityDenominator#143254]
(471) Project
Output [1]: [plant#143214]
Input [2]: [plant#143214, quantityDenominator#143254]
(472) HashAggregate
Input [1]: [plant#143214]
Keys [1]: [plant#143214]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143214]
(473) Exchange
Input [1]: [plant#143214]
Arguments: hashpartitioning(plant#143214, 25), ENSURE_REQUIREMENTS, [plan_id=217225]
(474) HashAggregate
Input [1]: [plant#143214]
Keys [1]: [plant#143214]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143214]
(475) BroadcastHashJoin
Left keys [1]: [plantUuid#143210]
Right keys [1]: [plant#143214]
Join type: Inner
Join condition: None
(476) Project
Output [3]: [plantUuid#143210, storageLocId#143211, storageLocUuid#143212]
Input [4]: [plantUuid#143210, storageLocId#143211, storageLocUuid#143212, plant#143214]
(477) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143182.internalRefUUID, ), isnull(plant#143182.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#143210, ), isnull(plantUuid#143210)]
Join type: Inner
Join condition: None
(478) Generate
Input [9]: [systemId#143179, internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, stockQuantities#143184, plantUuid#143210, storageLocId#143211, storageLocUuid#143212]
Arguments: explode(filter(stockQuantities#143184, lambdafunction(((lambda x#143085.storageLocation.internalRefUuid = storageLocUuid#143212) AND (lambda x#143085.storageLocation.Id = storageLocId#143211)), lambda x#143085, false))), [systemId#143179, internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, plantUuid#143210], false, [stockQuantity#143257]
(479) ObjectHashAggregate
Input [7]: [systemId#143179, internalUUID#143180, product#143181, plant#143182, calculationDateTime#143183, plantUuid#143210, stockQuantity#143257]
Keys [5]: [plantUuid#143210, internalUUID#143180, plant#143182, product#143181, systemId#143179]
Functions [2]: [partial_first(calculationDateTime#143183, false), partial_collect_list(stockQuantity#143257, 0, 0)]
Aggregate Attributes [3]: [first#144820, valueSet#144821, buf#144822]
Results [8]: [plantUuid#143210, internalUUID#143180, plant#143182, product#143181, systemId#143179, first#144823, valueSet#144824, buf#144825]
(480) ObjectHashAggregate
Input [8]: [plantUuid#143210, internalUUID#143180, plant#143182, product#143181, systemId#143179, first#144823, valueSet#144824, buf#144825]
Keys [5]: [plantUuid#143210, internalUUID#143180, plant#143182, product#143181, systemId#143179]
Functions [2]: [first(calculationDateTime#143183, false), collect_list(stockQuantity#143257, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#143183)()#143100, collect_list(stockQuantity#143257, 0, 0)#143101]
Results [4]: [product#143181, plant#143182, first(calculationDateTime#143183)()#143100 AS calculationDateTime#143258, collect_list(stockQuantity#143257, 0, 0)#143101 AS stockQuantities#143259]
(481) Filter
Input [4]: [product#143181, plant#143182, calculationDateTime#143258, stockQuantities#143259]
Condition : ((size(stockQuantities#143259, true) > 0) AND isnotnull(calculationDateTime#143258))
(482) Generate
Input [4]: [product#143181, plant#143182, calculationDateTime#143258, stockQuantities#143259]
Arguments: explode(stockQuantities#143259), [product#143181, plant#143182, calculationDateTime#143258], false, [stockQuantity#143260]
(483) Filter
Input [4]: [product#143181, plant#143182, calculationDateTime#143258, stockQuantity#143260]
Condition : (((isnotnull(stockQuantity#143260.specialStockIndicator.code) AND isnotnull(stockQuantity#143260.stockType.code)) AND (stockQuantity#143260.specialStockIndicator.code = )) AND (stockQuantity#143260.stockType.code = 01))
(484) Project
Output [4]: [plant#143182.internalRefUUID AS plantUuid#143262, product#143181.internalRefUUID AS productUuid#143264, calculationDateTime#143258, stockQuantity#143260.storagelocation.id AS storageLocId#143265]
Input [4]: [product#143181, plant#143182, calculationDateTime#143258, stockQuantity#143260]
(485) Exchange
Input [4]: [plantUuid#143262, productUuid#143264, calculationDateTime#143258, storageLocId#143265]
Arguments: hashpartitioning(productUuid#143264, plantUuid#143262, 25), ENSURE_REQUIREMENTS, [plan_id=217258]
(486) Sort
Input [4]: [plantUuid#143262, productUuid#143264, calculationDateTime#143258, storageLocId#143265]
Arguments: [productUuid#143264 ASC NULLS FIRST, plantUuid#143262 ASC NULLS FIRST], false, 0
(487) Scan parquet
Output [3]: [product#143270, plant#143271, orderDateTime#143272]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(488) Filter
Input [3]: [product#143270, plant#143271, orderDateTime#143272]
Condition : ((isnotnull(orderDateTime#143272) AND isnotnull(plant#143271)) AND isnotnull(product#143270))
(489) Exchange
Input [3]: [product#143270, plant#143271, orderDateTime#143272]
Arguments: hashpartitioning(product#143270, plant#143271, 25), REPARTITION_BY_NUM, [plan_id=217009]
(490) Scan parquet
Output [4]: [plant#143276, product#143277, productionAspect#143279, systemId#143274]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(491) Project
Output [4]: [plant#143276.internalRefUUID AS _extract_internalRefUUID#144147, product#143277.internalRefUUID AS _extract_internalRefUUID#144146, productionAspect#143279.productMovementPlants.unitOfIssue.code AS _extract_code#144144, productionAspect#143279.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144145]
Input [4]: [plant#143276, product#143277, productionAspect#143279, systemId#143274]
(492) Exchange
Input [4]: [_extract_internalRefUUID#144147, _extract_internalRefUUID#144146, _extract_code#144144, _extract_internalRefUUID#144145]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216662]
(493) Project
Output [4]: [_extract_code#144144, _extract_internalRefUUID#144145, _extract_internalRefUUID#144146, _extract_internalRefUUID#144147]
Input [4]: [_extract_internalRefUUID#144147, _extract_internalRefUUID#144146, _extract_code#144144, _extract_internalRefUUID#144145]
(494) BroadcastExchange
Input [4]: [_extract_code#144144, _extract_internalRefUUID#144145, _extract_internalRefUUID#144146, _extract_internalRefUUID#144147]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217241]
(495) BroadcastHashJoin
Left keys [4]: [coalesce(product#143270, ), isnull(product#143270), coalesce(plant#143271, ), isnull(plant#143271)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144146, ), isnull(_extract_internalRefUUID#144146), coalesce(_extract_internalRefUUID#144147, ), isnull(_extract_internalRefUUID#144147)]
Join type: LeftOuter
Join condition: None
(496) Project
Output [5]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145]
Input [7]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145, _extract_internalRefUUID#144146, _extract_internalRefUUID#144147]
(497) Scan parquet
Output [2]: [internalUUID#143284, systemId#143283]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(498) Project
Output [1]: [internalUUID#143284]
Input [2]: [internalUUID#143284, systemId#143283]
(499) Exchange
Input [1]: [internalUUID#143284]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216668]
(500) BroadcastExchange
Input [1]: [internalUUID#143284]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217245]
(501) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143271, ), isnull(plant#143271)]
Right keys [2]: [coalesce(internalUUID#143284, ), isnull(internalUUID#143284)]
Join type: LeftOuter
Join condition: None
(502) Project
Output [5]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145]
Input [6]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145, internalUUID#143284]
(503) Scan parquet
Output [4]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304, systemId#143297]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(504) Filter
Input [4]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304, systemId#143297]
Condition : isnotnull(internalUUID#143298)
(505) Project
Output [3]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
Input [4]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304, systemId#143297]
(506) Exchange
Input [3]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216674]
(507) BroadcastExchange
Input [3]: [internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217249]
(508) BroadcastHashJoin
Left keys [1]: [product#143270]
Right keys [1]: [internalUUID#143298]
Join type: LeftOuter
Join condition: None
(509) Project
Output [4]: [product#143270, plant#143271, orderDateTime#143272, filter(unitOfMeasureConversions#143304, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144144) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144145)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143302.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143302.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143309]
Input [8]: [product#143270, plant#143271, orderDateTime#143272, _extract_code#144144, _extract_internalRefUUID#144145, internalUUID#143298, baseUnitOfMeasure#143302, unitOfMeasureConversions#143304]
(510) Generate
Input [4]: [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309]
Arguments: explode(filteredUnitOfMeasureConversions#143309.quantityNumerator), [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309], true, [quantityNumerator#143310]
(511) Project
Output [4]: [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309.quantityDenominator AS _extract_quantityDenominator#143954]
Input [5]: [product#143270, plant#143271, orderDateTime#143272, filteredUnitOfMeasureConversions#143309, quantityNumerator#143310]
(512) Generate
Input [4]: [product#143270, plant#143271, orderDateTime#143272, _extract_quantityDenominator#143954]
Arguments: explode(_extract_quantityDenominator#143954), [product#143270, plant#143271, orderDateTime#143272], true, [quantityDenominator#143311]
(513) Project
Output [3]: [product#143270, plant#143271, orderDateTime#143272]
Input [4]: [product#143270, plant#143271, orderDateTime#143272, quantityDenominator#143311]
(514) Sort
Input [3]: [product#143270, plant#143271, orderDateTime#143272]
Arguments: [product#143270 ASC NULLS FIRST, plant#143271 ASC NULLS FIRST], false, 0
(515) SortMergeJoin
Left keys [2]: [productUuid#143264, plantUuid#143262]
Right keys [2]: [product#143270, plant#143271]
Join type: Inner
Join condition: (calculationDateTime#143258 <= orderDateTime#143272)
(516) Project
Output [4]: [plantUuid#143262 AS plant#143314, productUuid#143264 AS product#143315, storageLocId#143265, calculationDateTime#143258]
Input [7]: [plantUuid#143262, productUuid#143264, calculationDateTime#143258, storageLocId#143265, product#143270, plant#143271, orderDateTime#143272]
(517) HashAggregate
Input [4]: [plant#143314, product#143315, storageLocId#143265, calculationDateTime#143258]
Keys [3]: [plant#143314, product#143315, storageLocId#143265]
Functions [1]: [partial_max(calculationDateTime#143258)]
Aggregate Attributes [1]: [max#144818]
Results [4]: [plant#143314, product#143315, storageLocId#143265, max#144819]
(518) HashAggregate
Input [4]: [plant#143314, product#143315, storageLocId#143265, max#144819]
Keys [3]: [plant#143314, product#143315, storageLocId#143265]
Functions [1]: [max(calculationDateTime#143258)]
Aggregate Attributes [1]: [max(calculationDateTime#143258)#143178]
Results [4]: [plant#143314, product#143315, storageLocId#143265, max(calculationDateTime#143258)#143178 AS max_calc_datetime#143172]
(519) Exchange
Input [4]: [plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
Arguments: hashpartitioning(coalesce(plant#143314, ), isnull(plant#143314), coalesce(product#143315, ), isnull(product#143315), coalesce(storageLocId#143265, ), isnull(storageLocId#143265), coalesce(max_calc_datetime#143172, 1970-01-01 00:00:00), isnull(max_calc_datetime#143172), 25), ENSURE_REQUIREMENTS, [plan_id=217268]
(520) Sort
Input [4]: [plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
Arguments: [coalesce(plant#143314, ) ASC NULLS FIRST, isnull(plant#143314) ASC NULLS FIRST, coalesce(product#143315, ) ASC NULLS FIRST, isnull(product#143315) ASC NULLS FIRST, coalesce(storageLocId#143265, ) ASC NULLS FIRST, isnull(storageLocId#143265) ASC NULLS FIRST, coalesce(max_calc_datetime#143172, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(max_calc_datetime#143172) ASC NULLS FIRST], false, 0
(521) SortMergeJoin
Left keys [8]: [coalesce(plant#143170, ), isnull(plant#143170), coalesce(product#143171, ), isnull(product#143171), coalesce(storageLocId#143108, ), isnull(storageLocId#143108), coalesce(calculationDateTime#143087, 1970-01-01 00:00:00), isnull(calculationDateTime#143087)]
Right keys [8]: [coalesce(plant#143314, ), isnull(plant#143314), coalesce(product#143315, ), isnull(product#143315), coalesce(storageLocId#143265, ), isnull(storageLocId#143265), coalesce(max_calc_datetime#143172, 1970-01-01 00:00:00), isnull(max_calc_datetime#143172)]
Join type: Inner
Join condition: None
(522) Project
Output [3]: [plant#143170, product#143171, calculationDateTime#143087]
Input [8]: [plant#143170, product#143171, storageLocId#143108, calculationDateTime#143087, plant#143314, product#143315, storageLocId#143265, max_calc_datetime#143172]
(523) HashAggregate
Input [3]: [plant#143170, product#143171, calculationDateTime#143087]
Keys [2]: [product#143171, plant#143170]
Functions [1]: [partial_min(calculationDateTime#143087)]
Aggregate Attributes [1]: [min#144810]
Results [3]: [product#143171, plant#143170, min#144811]
(524) Exchange
Input [3]: [product#143171, plant#143170, min#144811]
Arguments: hashpartitioning(product#143171, plant#143170, 25), ENSURE_REQUIREMENTS, [plan_id=217275]
(525) HashAggregate
Input [3]: [product#143171, plant#143170, min#144811]
Keys [2]: [product#143171, plant#143170]
Functions [1]: [min(calculationDateTime#143087)]
Aggregate Attributes [1]: [min(calculationDateTime#143087)#143334]
Results [3]: [product#143171, plant#143170, min(calculationDateTime#143087)#143334 AS minCalculationDateTime#143327]
(526) Sort
Input [3]: [product#143171, plant#143170, minCalculationDateTime#143327]
Arguments: [product#143171 ASC NULLS FIRST, plant#143170 ASC NULLS FIRST], false, 0
(527) SortMergeJoin
Left keys [2]: [product#142980, plant#142981]
Right keys [2]: [product#143171, plant#143170]
Join type: LeftOuter
Join condition: None
(528) Project
Output [4]: [product#142980, plant#142981, timezonecode#143012, CASE WHEN isnotnull(minCalculationDateTime#143327) THEN minCalculationDateTime#143327 ELSE orderDateTime#142982 END AS stockDatetime#143384]
Input [7]: [product#142980, plant#142981, orderDateTime#142982, timezonecode#143012, product#143171, plant#143170, minCalculationDateTime#143327]
(529) Exchange
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: hashpartitioning(coalesce(plant#142981, ), isnull(plant#142981), coalesce(product#142980, ), isnull(product#142980), 25), ENSURE_REQUIREMENTS, [plan_id=217439]
(530) Sort
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: [coalesce(plant#142981, ) ASC NULLS FIRST, isnull(plant#142981) ASC NULLS FIRST, coalesce(product#142980, ) ASC NULLS FIRST, isnull(product#142980) ASC NULLS FIRST], false, 0
(531) Exchange
Input [3]: [product#143466, plant#143467, orderDateTime#143468]
Arguments: hashpartitioning(product#143466, plant#143467, 25), REPARTITION_BY_NUM, [plan_id=216692]
(532) Project
Output [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
Input [8]: [internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477, systemId#143470]
(533) Exchange
Input [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216695]
(534) Project
Output [1]: [struct(systemId, systemId#143470, internalUUID, internalUUID#143471, plant, plant#143472, product, product#143473, blockedForReplenishmentStartingFrom, blockedForReplenishmentStartingFrom#143474, productionAspect, productionAspect#143475, salesPlant, salesPlant#143476, listing, listing#143477) AS productPlant#143478]
Input [8]: [systemId#143470, internalUUID#143471, plant#143472, product#143473, blockedForReplenishmentStartingFrom#143474, productionAspect#143475, salesPlant#143476, listing#143477]
(535) BroadcastExchange
Input [1]: [productPlant#143478]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID), coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID)),false), [plan_id=217283]
(536) BroadcastHashJoin
Left keys [4]: [coalesce(product#143466, ), isnull(product#143466), coalesce(plant#143467, ), isnull(plant#143467)]
Right keys [4]: [coalesce(productPlant#143478.product.internalRefUUID, ), isnull(productPlant#143478.product.internalRefUUID), coalesce(productPlant#143478.plant.internalRefUUID, ), isnull(productPlant#143478.plant.internalRefUUID)]
Join type: LeftOuter
Join condition: None
(537) Scan parquet
Output [3]: [internalUUID#143480, address#143486, systemId#143479]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string,address:struct<timeZone:struct<code:string>>>
(538) Project
Output [2]: [internalUUID#143480, address#143486.timeZone.code AS timezonecode#143491]
Input [3]: [internalUUID#143480, address#143486, systemId#143479]
(539) Exchange
Input [2]: [internalUUID#143480, timezonecode#143491]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216700]
(540) BroadcastExchange
Input [2]: [internalUUID#143480, timezonecode#143491]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217286]
(541) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143467, ), isnull(plant#143467)]
Right keys [2]: [coalesce(internalUUID#143480, ), isnull(internalUUID#143480)]
Join type: LeftOuter
Join condition: None
(542) Project
Output [5]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, CASE WHEN isnotnull(timezonecode#143491) THEN timezonecode#143491 ELSE UTC END AS timezonecode#143492]
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, internalUUID#143480, timezonecode#143491]
(543) Scan parquet
Output [4]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500, systemId#143493]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(544) Filter
Input [4]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500, systemId#143493]
Condition : isnotnull(internalUUID#143494)
(545) Project
Output [3]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
Input [4]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500, systemId#143493]
(546) Exchange
Input [3]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216706]
(547) BroadcastExchange
Input [3]: [internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217290]
(548) BroadcastHashJoin
Left keys [1]: [product#143466]
Right keys [1]: [internalUUID#143494]
Join type: LeftOuter
Join condition: None
(549) Project
Output [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, filter(unitOfMeasureConversions#143500, lambdafunction((((lambda u#143027.measurementUnit1.code = productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.code) AND (lambda u#143027.measurementUnit1.internalRefUUID = productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143498.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143498.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143505]
Input [8]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, internalUUID#143494, baseUnitOfMeasure#143498, unitOfMeasureConversions#143500]
(550) Project
Output [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, filteredUnitOfMeasureConversions#143505.quantityDenominator AS _extract_quantityDenominator#144016, filteredUnitOfMeasureConversions#143505.quantityNumerator AS _extract_quantityNumerator#144017]
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, filteredUnitOfMeasureConversions#143505]
(551) Generate
Input [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, _extract_quantityDenominator#144016, _extract_quantityNumerator#144017]
Arguments: explode(_extract_quantityNumerator#144017), [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, _extract_quantityDenominator#144016], true, [quantityNumerator#143506]
(552) Generate
Input [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, _extract_quantityDenominator#144016, quantityNumerator#143506]
Arguments: explode(_extract_quantityDenominator#144016), [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, quantityNumerator#143506], true, [quantityDenominator#143507]
(553) Project
Output [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, (cast(quantityNumerator#143506 as double) / cast(quantityDenominator#143507 as double)) AS outboundUnit#143508]
Input [7]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, quantityNumerator#143506, quantityDenominator#143507]
(554) Project
Output [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, CASE WHEN (isnull(outboundUnit#143508) OR (outboundUnit#143508 <= 0.0)) THEN 1.0 ELSE outboundUnit#143508 END AS outboundUnit#143509]
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143508]
(555) Sort
Input [6]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509]
Arguments: [product#143466 ASC NULLS FIRST, plant#143467 ASC NULLS FIRST], false, 0
(556) Scan parquet
Output [6]: [internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515, systemId#143510]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(557) Filter
Input [6]: [internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515, systemId#143510]
Condition : (isnotnull(plant#143513.internalRefUUID) AND isnotnull(product#143512.internalRefUUID))
(558) Project
Output [6]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515]
Input [6]: [internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515, systemId#143510]
(559) Exchange
Input [6]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216717]
(560) BroadcastExchange
Input [6]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=217322]
(561) Scan parquet
Output [2]: [marketUnit#143516, distributionCenters#143522]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(562) Filter
Input [2]: [marketUnit#143516, distributionCenters#143522]
Condition : (((marketUnit#143516 <=> Daily_grocery) AND (size(distributionCenters#143522, true) > 0)) AND isnotnull(distributionCenters#143522))
(563) Project
Output [1]: [distributionCenters#143522]
Input [2]: [marketUnit#143516, distributionCenters#143522]
(564) Exchange
Input [1]: [distributionCenters#143522]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216721]
(565) Generate
Input [1]: [distributionCenters#143522]
Arguments: explode(distributionCenters#143522), false, [distributionCenters#143538]
(566) Filter
Input [1]: [distributionCenters#143538]
Condition : (((size(distributionCenters#143538.storageLocations, true) > 0) AND isnotnull(distributionCenters#143538.storageLocations)) AND isnotnull(distributionCenters#143538.internalRefUUID))
(567) Project
Output [2]: [distributionCenters#143538.internalRefUUID AS _extract_internalRefUUID#143960, distributionCenters#143538.storageLocations AS _extract_storageLocations#143961]
Input [1]: [distributionCenters#143538]
(568) Generate
Input [2]: [_extract_internalRefUUID#143960, _extract_storageLocations#143961]
Arguments: explode(_extract_storageLocations#143961), [_extract_internalRefUUID#143960], false, [storageLocations#143539]
(569) Project
Output [3]: [_extract_internalRefUUID#143960 AS plantUuid#143541, storageLocations#143539.id AS storageLocId#143542, storageLocations#143539.internalRefUUID AS storageLocUuid#143543]
Input [2]: [_extract_internalRefUUID#143960, storageLocations#143539]
(570) BroadcastExchange
Input [3]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217318]
(571) Scan parquet
Output [2]: [product#143544, plant#143545]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(572) Filter
Input [2]: [product#143544, plant#143545]
Condition : isnotnull(plant#143545)
(573) Exchange
Input [2]: [product#143544, plant#143545]
Arguments: hashpartitioning(product#143544, plant#143545, 25), REPARTITION_BY_NUM, [plan_id=217030]
(574) Scan parquet
Output [4]: [plant#143550, product#143551, productionAspect#143553, systemId#143548]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(575) Project
Output [4]: [plant#143550.internalRefUUID AS _extract_internalRefUUID#144148, product#143551.internalRefUUID AS _extract_internalRefUUID#144149, productionAspect#143553.productMovementPlants.unitOfIssue.code AS _extract_code#144150, productionAspect#143553.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144151]
Input [4]: [plant#143550, product#143551, productionAspect#143553, systemId#143548]
(576) Exchange
Input [4]: [_extract_internalRefUUID#144148, _extract_internalRefUUID#144149, _extract_code#144150, _extract_internalRefUUID#144151]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216733]
(577) Project
Output [4]: [_extract_code#144150, _extract_internalRefUUID#144151, _extract_internalRefUUID#144149, _extract_internalRefUUID#144148]
Input [4]: [_extract_internalRefUUID#144148, _extract_internalRefUUID#144149, _extract_code#144150, _extract_internalRefUUID#144151]
(578) BroadcastExchange
Input [4]: [_extract_code#144150, _extract_internalRefUUID#144151, _extract_internalRefUUID#144149, _extract_internalRefUUID#144148]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217298]
(579) BroadcastHashJoin
Left keys [4]: [coalesce(product#143544, ), isnull(product#143544), coalesce(plant#143545, ), isnull(plant#143545)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144149, ), isnull(_extract_internalRefUUID#144149), coalesce(_extract_internalRefUUID#144148, ), isnull(_extract_internalRefUUID#144148)]
Join type: LeftOuter
Join condition: None
(580) Project
Output [4]: [product#143544, plant#143545, _extract_code#144150, _extract_internalRefUUID#144151]
Input [6]: [product#143544, plant#143545, _extract_code#144150, _extract_internalRefUUID#144151, _extract_internalRefUUID#144149, _extract_internalRefUUID#144148]
(581) Scan parquet
Output [2]: [internalUUID#143558, systemId#143557]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(582) Project
Output [1]: [internalUUID#143558]
Input [2]: [internalUUID#143558, systemId#143557]
(583) Exchange
Input [1]: [internalUUID#143558]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216739]
(584) BroadcastExchange
Input [1]: [internalUUID#143558]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217302]
(585) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143545, ), isnull(plant#143545)]
Right keys [2]: [coalesce(internalUUID#143558, ), isnull(internalUUID#143558)]
Join type: LeftOuter
Join condition: None
(586) Project
Output [4]: [product#143544, plant#143545, _extract_code#144150, _extract_internalRefUUID#144151]
Input [5]: [product#143544, plant#143545, _extract_code#144150, _extract_internalRefUUID#144151, internalUUID#143558]
(587) Scan parquet
Output [4]: [internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578, systemId#143571]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(588) Filter
Input [4]: [internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578, systemId#143571]
Condition : isnotnull(internalUUID#143572)
(589) Project
Output [3]: [internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578]
Input [4]: [internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578, systemId#143571]
(590) Exchange
Input [3]: [internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216745]
(591) BroadcastExchange
Input [3]: [internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217306]
(592) BroadcastHashJoin
Left keys [1]: [product#143544]
Right keys [1]: [internalUUID#143572]
Join type: LeftOuter
Join condition: None
(593) Project
Output [2]: [plant#143545, filter(unitOfMeasureConversions#143578, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144150) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144151)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143576.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143576.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143583]
Input [7]: [product#143544, plant#143545, _extract_code#144150, _extract_internalRefUUID#144151, internalUUID#143572, baseUnitOfMeasure#143576, unitOfMeasureConversions#143578]
(594) Generate
Input [2]: [plant#143545, filteredUnitOfMeasureConversions#143583]
Arguments: explode(filteredUnitOfMeasureConversions#143583.quantityNumerator), [plant#143545, filteredUnitOfMeasureConversions#143583], true, [quantityNumerator#143584]
(595) Project
Output [2]: [plant#143545, filteredUnitOfMeasureConversions#143583.quantityDenominator AS _extract_quantityDenominator#143962]
Input [3]: [plant#143545, filteredUnitOfMeasureConversions#143583, quantityNumerator#143584]
(596) Generate
Input [2]: [plant#143545, _extract_quantityDenominator#143962]
Arguments: explode(_extract_quantityDenominator#143962), [plant#143545], true, [quantityDenominator#143585]
(597) Project
Output [1]: [plant#143545]
Input [2]: [plant#143545, quantityDenominator#143585]
(598) HashAggregate
Input [1]: [plant#143545]
Keys [1]: [plant#143545]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143545]
(599) Exchange
Input [1]: [plant#143545]
Arguments: hashpartitioning(plant#143545, 25), ENSURE_REQUIREMENTS, [plan_id=217315]
(600) HashAggregate
Input [1]: [plant#143545]
Keys [1]: [plant#143545]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143545]
(601) BroadcastHashJoin
Left keys [1]: [plantUuid#143541]
Right keys [1]: [plant#143545]
Join type: Inner
Join condition: None
(602) Project
Output [3]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
Input [4]: [plantUuid#143541, storageLocId#143542, storageLocUuid#143543, plant#143545]
(603) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143513.internalRefUUID, ), isnull(plant#143513.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#143541, ), isnull(plantUuid#143541)]
Join type: Inner
Join condition: None
(604) Generate
Input [9]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, stockQuantities#143515, plantUuid#143541, storageLocId#143542, storageLocUuid#143543]
Arguments: explode(filter(stockQuantities#143515, lambdafunction(((lambda x#143085.storageLocation.internalRefUuid = storageLocUuid#143543) AND (lambda x#143085.storageLocation.Id = storageLocId#143542)), lambda x#143085, false))), [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, plantUuid#143541], false, [stockQuantity#143588]
(605) ObjectHashAggregate
Input [7]: [systemId#143510, internalUUID#143511, product#143512, plant#143513, calculationDateTime#143514, plantUuid#143541, stockQuantity#143588]
Keys [5]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510]
Functions [2]: [partial_first(calculationDateTime#143514, false), partial_collect_list(stockQuantity#143588, 0, 0)]
Aggregate Attributes [3]: [first#144834, valueSet#144835, buf#144836]
Results [8]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510, first#144837, valueSet#144838, buf#144839]
(606) ObjectHashAggregate
Input [8]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510, first#144837, valueSet#144838, buf#144839]
Keys [5]: [plantUuid#143541, internalUUID#143511, plant#143513, product#143512, systemId#143510]
Functions [2]: [first(calculationDateTime#143514, false), collect_list(stockQuantity#143588, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#143514)()#143100, collect_list(stockQuantity#143588, 0, 0)#143101]
Results [4]: [product#143512, plant#143513, first(calculationDateTime#143514)()#143100 AS calculationDateTime#143589, collect_list(stockQuantity#143588, 0, 0)#143101 AS stockQuantities#143590]
(607) Filter
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantities#143590]
Condition : ((size(stockQuantities#143590, true) > 0) AND isnotnull(calculationDateTime#143589))
(608) Generate
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantities#143590]
Arguments: explode(stockQuantities#143590), [product#143512, plant#143513, calculationDateTime#143589], false, [stockQuantity#143591]
(609) Filter
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantity#143591]
Condition : (((isnotnull(stockQuantity#143591.specialStockIndicator.code) AND isnotnull(stockQuantity#143591.stockType.code)) AND (stockQuantity#143591.specialStockIndicator.code = )) AND (stockQuantity#143591.stockType.code = 01))
(610) Project
Output [5]: [plant#143513.internalRefUUID AS plantUuid#143593, product#143512.internalRefUUID AS productUuid#143595, calculationDateTime#143589, stockQuantity#143591.storagelocation.id AS storageLocId#143596, stockQuantity#143591.quantity.measure AS quantity#143598]
Input [4]: [product#143512, plant#143513, calculationDateTime#143589, stockQuantity#143591]
(611) Exchange
Input [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: hashpartitioning(productUuid#143595, plantUuid#143593, 25), ENSURE_REQUIREMENTS, [plan_id=217348]
(612) Sort
Input [5]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598]
Arguments: [productUuid#143595 ASC NULLS FIRST, plantUuid#143593 ASC NULLS FIRST], false, 0
(613) Scan parquet
Output [3]: [product#143601, plant#143602, orderDateTime#143603]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(614) Filter
Input [3]: [product#143601, plant#143602, orderDateTime#143603]
Condition : ((isnotnull(orderDateTime#143603) AND isnotnull(plant#143602)) AND isnotnull(product#143601))
(615) Exchange
Input [3]: [product#143601, plant#143602, orderDateTime#143603]
Arguments: hashpartitioning(product#143601, plant#143602, 25), REPARTITION_BY_NUM, [plan_id=217053]
(616) Scan parquet
Output [4]: [plant#143607, product#143608, productionAspect#143610, systemId#143605]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(617) Project
Output [4]: [plant#143607.internalRefUUID AS _extract_internalRefUUID#144153, product#143608.internalRefUUID AS _extract_internalRefUUID#144152, productionAspect#143610.productMovementPlants.unitOfIssue.code AS _extract_code#144154, productionAspect#143610.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144155]
Input [4]: [plant#143607, product#143608, productionAspect#143610, systemId#143605]
(618) Exchange
Input [4]: [_extract_internalRefUUID#144153, _extract_internalRefUUID#144152, _extract_code#144154, _extract_internalRefUUID#144155]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216770]
(619) Project
Output [4]: [_extract_code#144154, _extract_internalRefUUID#144155, _extract_internalRefUUID#144152, _extract_internalRefUUID#144153]
Input [4]: [_extract_internalRefUUID#144153, _extract_internalRefUUID#144152, _extract_code#144154, _extract_internalRefUUID#144155]
(620) BroadcastExchange
Input [4]: [_extract_code#144154, _extract_internalRefUUID#144155, _extract_internalRefUUID#144152, _extract_internalRefUUID#144153]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217331]
(621) BroadcastHashJoin
Left keys [4]: [coalesce(product#143601, ), isnull(product#143601), coalesce(plant#143602, ), isnull(plant#143602)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144152, ), isnull(_extract_internalRefUUID#144152), coalesce(_extract_internalRefUUID#144153, ), isnull(_extract_internalRefUUID#144153)]
Join type: LeftOuter
Join condition: None
(622) Project
Output [5]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155]
Input [7]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155, _extract_internalRefUUID#144152, _extract_internalRefUUID#144153]
(623) Scan parquet
Output [2]: [internalUUID#143615, systemId#143614]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(624) Project
Output [1]: [internalUUID#143615]
Input [2]: [internalUUID#143615, systemId#143614]
(625) Exchange
Input [1]: [internalUUID#143615]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216776]
(626) BroadcastExchange
Input [1]: [internalUUID#143615]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217335]
(627) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143602, ), isnull(plant#143602)]
Right keys [2]: [coalesce(internalUUID#143615, ), isnull(internalUUID#143615)]
Join type: LeftOuter
Join condition: None
(628) Project
Output [5]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155]
Input [6]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155, internalUUID#143615]
(629) Scan parquet
Output [4]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635, systemId#143628]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(630) Filter
Input [4]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635, systemId#143628]
Condition : isnotnull(internalUUID#143629)
(631) Project
Output [3]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
Input [4]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635, systemId#143628]
(632) Exchange
Input [3]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216782]
(633) BroadcastExchange
Input [3]: [internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217339]
(634) BroadcastHashJoin
Left keys [1]: [product#143601]
Right keys [1]: [internalUUID#143629]
Join type: LeftOuter
Join condition: None
(635) Project
Output [4]: [product#143601, plant#143602, orderDateTime#143603, filter(unitOfMeasureConversions#143635, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144154) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144155)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143633.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143633.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143640]
Input [8]: [product#143601, plant#143602, orderDateTime#143603, _extract_code#144154, _extract_internalRefUUID#144155, internalUUID#143629, baseUnitOfMeasure#143633, unitOfMeasureConversions#143635]
(636) Generate
Input [4]: [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640]
Arguments: explode(filteredUnitOfMeasureConversions#143640.quantityNumerator), [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640], true, [quantityNumerator#143641]
(637) Project
Output [4]: [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640.quantityDenominator AS _extract_quantityDenominator#143966]
Input [5]: [product#143601, plant#143602, orderDateTime#143603, filteredUnitOfMeasureConversions#143640, quantityNumerator#143641]
(638) Generate
Input [4]: [product#143601, plant#143602, orderDateTime#143603, _extract_quantityDenominator#143966]
Arguments: explode(_extract_quantityDenominator#143966), [product#143601, plant#143602, orderDateTime#143603], true, [quantityDenominator#143642]
(639) Project
Output [3]: [product#143601, plant#143602, orderDateTime#143603]
Input [4]: [product#143601, plant#143602, orderDateTime#143603, quantityDenominator#143642]
(640) Sort
Input [3]: [product#143601, plant#143602, orderDateTime#143603]
Arguments: [product#143601 ASC NULLS FIRST, plant#143602 ASC NULLS FIRST], false, 0
(641) SortMergeJoin
Left keys [2]: [productUuid#143595, plantUuid#143593]
Right keys [2]: [product#143601, plant#143602]
Join type: Inner
Join condition: (calculationDateTime#143589 <= orderDateTime#143603)
(642) Project
Output [5]: [plantUuid#143593 AS plant#143645, productUuid#143595 AS product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Input [8]: [plantUuid#143593, productUuid#143595, calculationDateTime#143589, storageLocId#143596, quantity#143598, product#143601, plant#143602, orderDateTime#143603]
(643) Exchange
Input [5]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Arguments: hashpartitioning(coalesce(plant#143645, ), isnull(plant#143645), coalesce(product#143646, ), isnull(product#143646), coalesce(storageLocId#143596, ), isnull(storageLocId#143596), coalesce(calculationDateTime#143589, 1970-01-01 00:00:00), isnull(calculationDateTime#143589), 25), ENSURE_REQUIREMENTS, [plan_id=217412]
(644) Sort
Input [5]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598]
Arguments: [coalesce(plant#143645, ) ASC NULLS FIRST, isnull(plant#143645) ASC NULLS FIRST, coalesce(product#143646, ) ASC NULLS FIRST, isnull(product#143646) ASC NULLS FIRST, coalesce(storageLocId#143596, ) ASC NULLS FIRST, isnull(storageLocId#143596) ASC NULLS FIRST, coalesce(calculationDateTime#143589, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(calculationDateTime#143589) ASC NULLS FIRST], false, 0
(645) Scan parquet
Output [6]: [internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652, systemId#143647]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(646) Filter
Input [6]: [internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652, systemId#143647]
Condition : (isnotnull(plant#143650.internalRefUUID) AND isnotnull(product#143649.internalRefUUID))
(647) Project
Output [6]: [systemId#143647, internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652]
Input [6]: [internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652, systemId#143647]
(648) Exchange
Input [6]: [systemId#143647, internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216794]
(649) BroadcastExchange
Input [6]: [systemId#143647, internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=217377]
(650) Scan parquet
Output [2]: [marketUnit#143653, distributionCenters#143659]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(651) Filter
Input [2]: [marketUnit#143653, distributionCenters#143659]
Condition : (((marketUnit#143653 <=> Daily_grocery) AND (size(distributionCenters#143659, true) > 0)) AND isnotnull(distributionCenters#143659))
(652) Project
Output [1]: [distributionCenters#143659]
Input [2]: [marketUnit#143653, distributionCenters#143659]
(653) Exchange
Input [1]: [distributionCenters#143659]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216798]
(654) Generate
Input [1]: [distributionCenters#143659]
Arguments: explode(distributionCenters#143659), false, [distributionCenters#143675]
(655) Filter
Input [1]: [distributionCenters#143675]
Condition : (((size(distributionCenters#143675.storageLocations, true) > 0) AND isnotnull(distributionCenters#143675.storageLocations)) AND isnotnull(distributionCenters#143675.internalRefUUID))
(656) Project
Output [2]: [distributionCenters#143675.internalRefUUID AS _extract_internalRefUUID#143970, distributionCenters#143675.storageLocations AS _extract_storageLocations#143971]
Input [1]: [distributionCenters#143675]
(657) Generate
Input [2]: [_extract_internalRefUUID#143970, _extract_storageLocations#143971]
Arguments: explode(_extract_storageLocations#143971), [_extract_internalRefUUID#143970], false, [storageLocations#143676]
(658) Project
Output [3]: [_extract_internalRefUUID#143970 AS plantUuid#143678, storageLocations#143676.id AS storageLocId#143679, storageLocations#143676.internalRefUUID AS storageLocUuid#143680]
Input [2]: [_extract_internalRefUUID#143970, storageLocations#143676]
(659) BroadcastExchange
Input [3]: [plantUuid#143678, storageLocId#143679, storageLocUuid#143680]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217373]
(660) Scan parquet
Output [2]: [product#143681, plant#143682]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(661) Filter
Input [2]: [product#143681, plant#143682]
Condition : isnotnull(plant#143682)
(662) Exchange
Input [2]: [product#143681, plant#143682]
Arguments: hashpartitioning(product#143681, plant#143682, 25), REPARTITION_BY_NUM, [plan_id=217066]
(663) Scan parquet
Output [4]: [plant#143687, product#143688, productionAspect#143690, systemId#143685]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(664) Project
Output [4]: [plant#143687.internalRefUUID AS _extract_internalRefUUID#144159, product#143688.internalRefUUID AS _extract_internalRefUUID#144156, productionAspect#143690.productMovementPlants.unitOfIssue.code AS _extract_code#144157, productionAspect#143690.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144158]
Input [4]: [plant#143687, product#143688, productionAspect#143690, systemId#143685]
(665) Exchange
Input [4]: [_extract_internalRefUUID#144159, _extract_internalRefUUID#144156, _extract_code#144157, _extract_internalRefUUID#144158]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216810]
(666) Project
Output [4]: [_extract_code#144157, _extract_internalRefUUID#144158, _extract_internalRefUUID#144156, _extract_internalRefUUID#144159]
Input [4]: [_extract_internalRefUUID#144159, _extract_internalRefUUID#144156, _extract_code#144157, _extract_internalRefUUID#144158]
(667) BroadcastExchange
Input [4]: [_extract_code#144157, _extract_internalRefUUID#144158, _extract_internalRefUUID#144156, _extract_internalRefUUID#144159]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217353]
(668) BroadcastHashJoin
Left keys [4]: [coalesce(product#143681, ), isnull(product#143681), coalesce(plant#143682, ), isnull(plant#143682)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144156, ), isnull(_extract_internalRefUUID#144156), coalesce(_extract_internalRefUUID#144159, ), isnull(_extract_internalRefUUID#144159)]
Join type: LeftOuter
Join condition: None
(669) Project
Output [4]: [product#143681, plant#143682, _extract_code#144157, _extract_internalRefUUID#144158]
Input [6]: [product#143681, plant#143682, _extract_code#144157, _extract_internalRefUUID#144158, _extract_internalRefUUID#144156, _extract_internalRefUUID#144159]
(670) Scan parquet
Output [2]: [internalUUID#143695, systemId#143694]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(671) Project
Output [1]: [internalUUID#143695]
Input [2]: [internalUUID#143695, systemId#143694]
(672) Exchange
Input [1]: [internalUUID#143695]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216816]
(673) BroadcastExchange
Input [1]: [internalUUID#143695]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217357]
(674) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143682, ), isnull(plant#143682)]
Right keys [2]: [coalesce(internalUUID#143695, ), isnull(internalUUID#143695)]
Join type: LeftOuter
Join condition: None
(675) Project
Output [4]: [product#143681, plant#143682, _extract_code#144157, _extract_internalRefUUID#144158]
Input [5]: [product#143681, plant#143682, _extract_code#144157, _extract_internalRefUUID#144158, internalUUID#143695]
(676) Scan parquet
Output [4]: [internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715, systemId#143708]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(677) Filter
Input [4]: [internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715, systemId#143708]
Condition : isnotnull(internalUUID#143709)
(678) Project
Output [3]: [internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715]
Input [4]: [internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715, systemId#143708]
(679) Exchange
Input [3]: [internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216822]
(680) BroadcastExchange
Input [3]: [internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217361]
(681) BroadcastHashJoin
Left keys [1]: [product#143681]
Right keys [1]: [internalUUID#143709]
Join type: LeftOuter
Join condition: None
(682) Project
Output [2]: [plant#143682, filter(unitOfMeasureConversions#143715, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144157) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144158)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143713.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143713.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143720]
Input [7]: [product#143681, plant#143682, _extract_code#144157, _extract_internalRefUUID#144158, internalUUID#143709, baseUnitOfMeasure#143713, unitOfMeasureConversions#143715]
(683) Generate
Input [2]: [plant#143682, filteredUnitOfMeasureConversions#143720]
Arguments: explode(filteredUnitOfMeasureConversions#143720.quantityNumerator), [plant#143682, filteredUnitOfMeasureConversions#143720], true, [quantityNumerator#143721]
(684) Project
Output [2]: [plant#143682, filteredUnitOfMeasureConversions#143720.quantityDenominator AS _extract_quantityDenominator#143972]
Input [3]: [plant#143682, filteredUnitOfMeasureConversions#143720, quantityNumerator#143721]
(685) Generate
Input [2]: [plant#143682, _extract_quantityDenominator#143972]
Arguments: explode(_extract_quantityDenominator#143972), [plant#143682], true, [quantityDenominator#143722]
(686) Project
Output [1]: [plant#143682]
Input [2]: [plant#143682, quantityDenominator#143722]
(687) HashAggregate
Input [1]: [plant#143682]
Keys [1]: [plant#143682]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143682]
(688) Exchange
Input [1]: [plant#143682]
Arguments: hashpartitioning(plant#143682, 25), ENSURE_REQUIREMENTS, [plan_id=217370]
(689) HashAggregate
Input [1]: [plant#143682]
Keys [1]: [plant#143682]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143682]
(690) BroadcastHashJoin
Left keys [1]: [plantUuid#143678]
Right keys [1]: [plant#143682]
Join type: Inner
Join condition: None
(691) Project
Output [3]: [plantUuid#143678, storageLocId#143679, storageLocUuid#143680]
Input [4]: [plantUuid#143678, storageLocId#143679, storageLocUuid#143680, plant#143682]
(692) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143650.internalRefUUID, ), isnull(plant#143650.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#143678, ), isnull(plantUuid#143678)]
Join type: Inner
Join condition: None
(693) Generate
Input [9]: [systemId#143647, internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, stockQuantities#143652, plantUuid#143678, storageLocId#143679, storageLocUuid#143680]
Arguments: explode(filter(stockQuantities#143652, lambdafunction(((lambda x#143085.storageLocation.internalRefUuid = storageLocUuid#143680) AND (lambda x#143085.storageLocation.Id = storageLocId#143679)), lambda x#143085, false))), [systemId#143647, internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, plantUuid#143678], false, [stockQuantity#143725]
(694) ObjectHashAggregate
Input [7]: [systemId#143647, internalUUID#143648, product#143649, plant#143650, calculationDateTime#143651, plantUuid#143678, stockQuantity#143725]
Keys [5]: [plantUuid#143678, internalUUID#143648, plant#143650, product#143649, systemId#143647]
Functions [2]: [partial_first(calculationDateTime#143651, false), partial_collect_list(stockQuantity#143725, 0, 0)]
Aggregate Attributes [3]: [first#144842, valueSet#144843, buf#144844]
Results [8]: [plantUuid#143678, internalUUID#143648, plant#143650, product#143649, systemId#143647, first#144845, valueSet#144846, buf#144847]
(695) ObjectHashAggregate
Input [8]: [plantUuid#143678, internalUUID#143648, plant#143650, product#143649, systemId#143647, first#144845, valueSet#144846, buf#144847]
Keys [5]: [plantUuid#143678, internalUUID#143648, plant#143650, product#143649, systemId#143647]
Functions [2]: [first(calculationDateTime#143651, false), collect_list(stockQuantity#143725, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#143651)()#143100, collect_list(stockQuantity#143725, 0, 0)#143101]
Results [4]: [product#143649, plant#143650, first(calculationDateTime#143651)()#143100 AS calculationDateTime#143726, collect_list(stockQuantity#143725, 0, 0)#143101 AS stockQuantities#143727]
(696) Filter
Input [4]: [product#143649, plant#143650, calculationDateTime#143726, stockQuantities#143727]
Condition : ((size(stockQuantities#143727, true) > 0) AND isnotnull(calculationDateTime#143726))
(697) Generate
Input [4]: [product#143649, plant#143650, calculationDateTime#143726, stockQuantities#143727]
Arguments: explode(stockQuantities#143727), [product#143649, plant#143650, calculationDateTime#143726], false, [stockQuantity#143728]
(698) Filter
Input [4]: [product#143649, plant#143650, calculationDateTime#143726, stockQuantity#143728]
Condition : (((isnotnull(stockQuantity#143728.specialStockIndicator.code) AND isnotnull(stockQuantity#143728.stockType.code)) AND (stockQuantity#143728.specialStockIndicator.code = )) AND (stockQuantity#143728.stockType.code = 01))
(699) Project
Output [4]: [plant#143650.internalRefUUID AS plantUuid#143730, product#143649.internalRefUUID AS productUuid#143732, calculationDateTime#143726, stockQuantity#143728.storagelocation.id AS storageLocId#143733]
Input [4]: [product#143649, plant#143650, calculationDateTime#143726, stockQuantity#143728]
(700) Exchange
Input [4]: [plantUuid#143730, productUuid#143732, calculationDateTime#143726, storageLocId#143733]
Arguments: hashpartitioning(productUuid#143732, plantUuid#143730, 25), ENSURE_REQUIREMENTS, [plan_id=217403]
(701) Sort
Input [4]: [plantUuid#143730, productUuid#143732, calculationDateTime#143726, storageLocId#143733]
Arguments: [productUuid#143732 ASC NULLS FIRST, plantUuid#143730 ASC NULLS FIRST], false, 0
(702) Scan parquet
Output [3]: [product#143738, plant#143739, orderDateTime#143740]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(703) Filter
Input [3]: [product#143738, plant#143739, orderDateTime#143740]
Condition : ((isnotnull(orderDateTime#143740) AND isnotnull(plant#143739)) AND isnotnull(product#143738))
(704) Exchange
Input [3]: [product#143738, plant#143739, orderDateTime#143740]
Arguments: hashpartitioning(product#143738, plant#143739, 25), REPARTITION_BY_NUM, [plan_id=217089]
(705) Scan parquet
Output [4]: [plant#143744, product#143745, productionAspect#143747, systemId#143742]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(706) Project
Output [4]: [plant#143744.internalRefUUID AS _extract_internalRefUUID#144161, product#143745.internalRefUUID AS _extract_internalRefUUID#144160, productionAspect#143747.productMovementPlants.unitOfIssue.code AS _extract_code#144162, productionAspect#143747.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144163]
Input [4]: [plant#143744, product#143745, productionAspect#143747, systemId#143742]
(707) Exchange
Input [4]: [_extract_internalRefUUID#144161, _extract_internalRefUUID#144160, _extract_code#144162, _extract_internalRefUUID#144163]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216847]
(708) Project
Output [4]: [_extract_code#144162, _extract_internalRefUUID#144163, _extract_internalRefUUID#144160, _extract_internalRefUUID#144161]
Input [4]: [_extract_internalRefUUID#144161, _extract_internalRefUUID#144160, _extract_code#144162, _extract_internalRefUUID#144163]
(709) BroadcastExchange
Input [4]: [_extract_code#144162, _extract_internalRefUUID#144163, _extract_internalRefUUID#144160, _extract_internalRefUUID#144161]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217386]
(710) BroadcastHashJoin
Left keys [4]: [coalesce(product#143738, ), isnull(product#143738), coalesce(plant#143739, ), isnull(plant#143739)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144160, ), isnull(_extract_internalRefUUID#144160), coalesce(_extract_internalRefUUID#144161, ), isnull(_extract_internalRefUUID#144161)]
Join type: LeftOuter
Join condition: None
(711) Project
Output [5]: [product#143738, plant#143739, orderDateTime#143740, _extract_code#144162, _extract_internalRefUUID#144163]
Input [7]: [product#143738, plant#143739, orderDateTime#143740, _extract_code#144162, _extract_internalRefUUID#144163, _extract_internalRefUUID#144160, _extract_internalRefUUID#144161]
(712) Scan parquet
Output [2]: [internalUUID#143752, systemId#143751]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(713) Project
Output [1]: [internalUUID#143752]
Input [2]: [internalUUID#143752, systemId#143751]
(714) Exchange
Input [1]: [internalUUID#143752]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216853]
(715) BroadcastExchange
Input [1]: [internalUUID#143752]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217390]
(716) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143739, ), isnull(plant#143739)]
Right keys [2]: [coalesce(internalUUID#143752, ), isnull(internalUUID#143752)]
Join type: LeftOuter
Join condition: None
(717) Project
Output [5]: [product#143738, plant#143739, orderDateTime#143740, _extract_code#144162, _extract_internalRefUUID#144163]
Input [6]: [product#143738, plant#143739, orderDateTime#143740, _extract_code#144162, _extract_internalRefUUID#144163, internalUUID#143752]
(718) Scan parquet
Output [4]: [internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772, systemId#143765]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(719) Filter
Input [4]: [internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772, systemId#143765]
Condition : isnotnull(internalUUID#143766)
(720) Project
Output [3]: [internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772]
Input [4]: [internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772, systemId#143765]
(721) Exchange
Input [3]: [internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216859]
(722) BroadcastExchange
Input [3]: [internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217394]
(723) BroadcastHashJoin
Left keys [1]: [product#143738]
Right keys [1]: [internalUUID#143766]
Join type: LeftOuter
Join condition: None
(724) Project
Output [4]: [product#143738, plant#143739, orderDateTime#143740, filter(unitOfMeasureConversions#143772, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144162) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144163)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143770.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143770.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143777]
Input [8]: [product#143738, plant#143739, orderDateTime#143740, _extract_code#144162, _extract_internalRefUUID#144163, internalUUID#143766, baseUnitOfMeasure#143770, unitOfMeasureConversions#143772]
(725) Generate
Input [4]: [product#143738, plant#143739, orderDateTime#143740, filteredUnitOfMeasureConversions#143777]
Arguments: explode(filteredUnitOfMeasureConversions#143777.quantityNumerator), [product#143738, plant#143739, orderDateTime#143740, filteredUnitOfMeasureConversions#143777], true, [quantityNumerator#143778]
(726) Project
Output [4]: [product#143738, plant#143739, orderDateTime#143740, filteredUnitOfMeasureConversions#143777.quantityDenominator AS _extract_quantityDenominator#143976]
Input [5]: [product#143738, plant#143739, orderDateTime#143740, filteredUnitOfMeasureConversions#143777, quantityNumerator#143778]
(727) Generate
Input [4]: [product#143738, plant#143739, orderDateTime#143740, _extract_quantityDenominator#143976]
Arguments: explode(_extract_quantityDenominator#143976), [product#143738, plant#143739, orderDateTime#143740], true, [quantityDenominator#143779]
(728) Project
Output [3]: [product#143738, plant#143739, orderDateTime#143740]
Input [4]: [product#143738, plant#143739, orderDateTime#143740, quantityDenominator#143779]
(729) Sort
Input [3]: [product#143738, plant#143739, orderDateTime#143740]
Arguments: [product#143738 ASC NULLS FIRST, plant#143739 ASC NULLS FIRST], false, 0
(730) SortMergeJoin
Left keys [2]: [productUuid#143732, plantUuid#143730]
Right keys [2]: [product#143738, plant#143739]
Join type: Inner
Join condition: (calculationDateTime#143726 <= orderDateTime#143740)
(731) Project
Output [4]: [plantUuid#143730 AS plant#143782, productUuid#143732 AS product#143783, storageLocId#143733, calculationDateTime#143726]
Input [7]: [plantUuid#143730, productUuid#143732, calculationDateTime#143726, storageLocId#143733, product#143738, plant#143739, orderDateTime#143740]
(732) HashAggregate
Input [4]: [plant#143782, product#143783, storageLocId#143733, calculationDateTime#143726]
Keys [3]: [plant#143782, product#143783, storageLocId#143733]
Functions [1]: [partial_max(calculationDateTime#143726)]
Aggregate Attributes [1]: [max#144840]
Results [4]: [plant#143782, product#143783, storageLocId#143733, max#144841]
(733) HashAggregate
Input [4]: [plant#143782, product#143783, storageLocId#143733, max#144841]
Keys [3]: [plant#143782, product#143783, storageLocId#143733]
Functions [1]: [max(calculationDateTime#143726)]
Aggregate Attributes [1]: [max(calculationDateTime#143726)#143178]
Results [4]: [plant#143782, product#143783, storageLocId#143733, max(calculationDateTime#143726)#143178 AS max_calc_datetime#143784]
(734) Exchange
Input [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
Arguments: hashpartitioning(coalesce(plant#143782, ), isnull(plant#143782), coalesce(product#143783, ), isnull(product#143783), coalesce(storageLocId#143733, ), isnull(storageLocId#143733), coalesce(max_calc_datetime#143784, 1970-01-01 00:00:00), isnull(max_calc_datetime#143784), 25), ENSURE_REQUIREMENTS, [plan_id=217413]
(735) Sort
Input [4]: [plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
Arguments: [coalesce(plant#143782, ) ASC NULLS FIRST, isnull(plant#143782) ASC NULLS FIRST, coalesce(product#143783, ) ASC NULLS FIRST, isnull(product#143783) ASC NULLS FIRST, coalesce(storageLocId#143733, ) ASC NULLS FIRST, isnull(storageLocId#143733) ASC NULLS FIRST, coalesce(max_calc_datetime#143784, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(max_calc_datetime#143784) ASC NULLS FIRST], false, 0
(736) SortMergeJoin
Left keys [8]: [coalesce(plant#143645, ), isnull(plant#143645), coalesce(product#143646, ), isnull(product#143646), coalesce(storageLocId#143596, ), isnull(storageLocId#143596), coalesce(calculationDateTime#143589, 1970-01-01 00:00:00), isnull(calculationDateTime#143589)]
Right keys [8]: [coalesce(plant#143782, ), isnull(plant#143782), coalesce(product#143783, ), isnull(product#143783), coalesce(storageLocId#143733, ), isnull(storageLocId#143733), coalesce(max_calc_datetime#143784, 1970-01-01 00:00:00), isnull(max_calc_datetime#143784)]
Join type: Inner
Join condition: None
(737) Project
Output [4]: [plant#143645, product#143646, calculationDateTime#143589, quantity#143598]
Input [9]: [plant#143645, product#143646, storageLocId#143596, calculationDateTime#143589, quantity#143598, plant#143782, product#143783, storageLocId#143733, max_calc_datetime#143784]
(738) HashAggregate
Input [4]: [plant#143645, product#143646, calculationDateTime#143589, quantity#143598]
Keys [2]: [product#143646, plant#143645]
Functions [2]: [partial_min(calculationDateTime#143589), partial_sum(quantity#143598)]
Aggregate Attributes [2]: [min#144830, sum#144831]
Results [4]: [product#143646, plant#143645, min#144832, sum#144833]
(739) Exchange
Input [4]: [product#143646, plant#143645, min#144832, sum#144833]
Arguments: hashpartitioning(product#143646, plant#143645, 25), ENSURE_REQUIREMENTS, [plan_id=217420]
(740) HashAggregate
Input [4]: [product#143646, plant#143645, min#144832, sum#144833]
Keys [2]: [product#143646, plant#143645]
Functions [2]: [min(calculationDateTime#143589), sum(quantity#143598)]
Aggregate Attributes [2]: [min(calculationDateTime#143589)#143334, sum(quantity#143598)#143333]
Results [4]: [product#143646, plant#143645, min(calculationDateTime#143589)#143334 AS minCalculationDateTime#143786, round(sum(quantity#143598)#143333, 0) AS roundedQuantitySum#143787]
(741) Sort
Input [4]: [product#143646, plant#143645, minCalculationDateTime#143786, roundedQuantitySum#143787]
Arguments: [product#143646 ASC NULLS FIRST, plant#143645 ASC NULLS FIRST], false, 0
(742) SortMergeJoin
Left keys [2]: [product#143466, plant#143467]
Right keys [2]: [product#143646, plant#143645]
Join type: LeftOuter
Join condition: None
(743) Project
Output [8]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, CASE WHEN isnotnull(minCalculationDateTime#143786) THEN minCalculationDateTime#143786 ELSE orderDateTime#143468 END AS stockDatetime#143788, CASE WHEN isnotnull(roundedQuantitySum#143787) THEN array(roundedQuantitySum#143787) ELSE [0.0] END AS stockBatchQuantities#143789]
Input [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, product#143646, plant#143645, minCalculationDateTime#143786, roundedQuantitySum#143787]
(744) Filter
Input [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
Condition : (isnotnull(product#143387) AND isnotnull(plant#143388))
(745) BroadcastExchange
Input [9]: [product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
Arguments: HashedRelationBroadcastMode(List(input[0, string, false], input[1, string, false]),false), [plan_id=217429]
(746) BroadcastHashJoin
Left keys [2]: [product#143466, plant#143467]
Right keys [2]: [product#143387, plant#143388]
Join type: LeftOuter
Join condition: None
(747) Project
Output [9]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, CASE WHEN isnotnull(product#143387) THEN struct(demandChannel, demandChannel#143389, demandStream, demandStream#143390, considerVariance, considerVariance#143391, demandTimeBuckets, demandTimeBuckets#143392, demandPointInTimeStart, demandPointInTimeStart#143393, demandPointInTimeEnd, demandPointInTimeEnd#143394, demandPointInTime, demandPointInTime#143395) END AS rawDemands#143412]
Input [17]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, product#143387, plant#143388, demandChannel#143389, demandStream#143390, considerVariance#143391, demandTimeBuckets#143392, demandPointInTimeStart#143393, demandPointInTimeEnd#143394, demandPointInTime#143395]
(748) ObjectHashAggregate
Input [9]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, rawDemands#143412]
Keys [9]: [product#143466, plant#143467, orderDateTime#143468, knownfloatingpointnormalized(if (isnull(productPlant#143478)) null else named_struct(systemId, productPlant#143478.systemId, internalUUID, productPlant#143478.internalUUID, plant, productPlant#143478.plant, product, productPlant#143478.product, blockedForReplenishmentStartingFrom, productPlant#143478.blockedForReplenishmentStartingFrom, productionAspect, knownfloatingpointnormalized(if (isnull(productPlant#143478.productionAspect)) null else named_struct(productMovementPlants, productPlant#143478.productionAspect.productMovementPlants, productPlanningPlants, knownfloatingpointnormalized(if (isnull(productPlant#143478.productionAspect.productPlanningPlants)) null else named_struct(mrpType, productPlant#143478.productionAspect.productPlanningPlants.mrpType, abcIndicator, productPlant#143478.productionAspect.productPlanningPlants.abcIndicator, mrpController, productPlant#143478.productionAspect.productPlanningPlants.mrpController, minimumLotSize, knownfloatingpointnormalized(normalizenanandzero(productPlant#143478.productionAspect.productPlanningPlants.minimumLotSize)), minimumSafetyStock, knownfloatingpointnormalized(normalizenanandzero(productPlant#143478.productionAspect.productPlanningPlants.minimumSafetyStock)), logisticalRoundingProfile, productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile, sourceOfSupplyCategory, productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory)))), salesPlant, productPlant#143478.salesPlant, listing, productPlant#143478.listing)) AS productPlant#143478, timezonecode#143492, knownfloatingpointnormalized(normalizenanandzero(outboundUnit#143509)) AS outboundUnit#143509, stockDatetime#143788, knownfloatingpointnormalized(transform(stockBatchQuantities#143789, lambdafunction(knownfloatingpointnormalized(normalizenanandzero(lambda arg#144827)), lambda arg#144827, false))) AS stockBatchQuantities#143789, [7226582400000000] AS [7226582400000000]#144826]
Functions [1]: [partial_collect_list(rawDemands#143412, 0, 0)]
Aggregate Attributes [1]: [buf#144828]
Results [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000]#144826, buf#144829]
(749) ObjectHashAggregate
Input [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000]#144826, buf#144829]
Keys [9]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000]#144826]
Functions [1]: [collect_list(rawDemands#143412, 0, 0)]
Aggregate Attributes [1]: [collect_list(rawDemands#143412, 0, 0)#143431]
Results [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, [7226582400000000] AS stockBatchShelfLifeEndDatetimes#143790, collect_list(rawDemands#143412, 0, 0)#143431 AS rawDemands#143413]
(750) DeserializeToObject
Input [10]: [product#143466, plant#143467, orderDateTime#143468, productPlant#143478, timezonecode#143492, outboundUnit#143509, stockDatetime#143788, stockBatchQuantities#143789, stockBatchShelfLifeEndDatetimes#143790, rawDemands#143413]
Arguments: createexternalrow(invoke(product#143466.toString()), invoke(plant#143467.toString()), static_invoke(DateTimeUtils.toJavaTimestamp(orderDateTime#143468)), if (isnull(productPlant#143478)) null else createexternalrow(invoke(productPlant#143478.systemId.toString()), invoke(productPlant#143478.internalUUID.toString()), if (isnull(productPlant#143478.plant)) null else createexternalrow(invoke(productPlant#143478.plant.internalRefUUID.toString()), invoke(productPlant#143478.plant.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), if (isnull(productPlant#143478.product)) null else createexternalrow(invoke(productPlant#143478.product.internalRefUUID.toString()), invoke(productPlant#143478.product.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(DateTimeUtils.toJavaDate(productPlant#143478.blockedForReplenishmentStartingFrom)), if (isnull(productPlant#143478.productionAspect)) null else createexternalrow(if (isnull(productPlant#143478.productionAspect.productMovementPlants)) null else createexternalrow(if (isnull(productPlant#143478.productionAspect.productMovementPlants.unitOfIssue)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productMovementPlants.unitOfIssue.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants)) null else createexternalrow(if (isnull(productPlant#143478.productionAspect.productPlanningPlants.mrpType)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpType.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpType.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.abcIndicator)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.abcIndicator.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.abcIndicator.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.mrpController)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpController.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.mrpController.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(java.lang.Double.valueOf(productPlant#143478.productionAspect.productPlanningPlants.minimumLotSize)), static_invoke(java.lang.Double.valueOf(productPlant#143478.productionAspect.productPlanningPlants.minimumSafetyStock)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.logisticalRoundingProfile.logisticalRoundingProfile.toString()), StructField(internalRefUUID,StringType,true), StructField(logisticalRoundingProfile,StringType,true)), if (isnull(productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory)) null else createexternalrow(invoke(productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory.internalRefUUID.toString()), invoke(productPlant#143478.productionAspect.productPlanningPlants.sourceOfSupplyCategory.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(minimumLotSize,DoubleType,true), StructField(minimumSafetyStock,DoubleType,true), StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true), StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)), if (isnull(productPlant#143478.salesPlant)) null else createexternalrow(if (isnull(productPlant#143478.salesPlant.loadingGroup)) null else createexternalrow(invoke(productPlant#143478.salesPlant.loadingGroup.internalRefUUID.toString()), invoke(productPlant#143478.salesPlant.loadingGroup.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), mapobjects(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1), if (isnull(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validFrom)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validTo)), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).isListed)), StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true)), productPlant#143478.listing, Some(class scala.collection.mutable.ArraySeq)), StructField(systemId,StringType,true), StructField(internalUUID,StringType,true), StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(blockedForReplenishmentStartingFrom,DateType,true), StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true), StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)), invoke(timezonecode#143492.toString()), static_invoke(java.lang.Double.valueOf(outboundUnit#143509)), static_invoke(DateTimeUtils.toJavaTimestamp(stockDatetime#143788)), mapobjects(lambdavariable(MapObject, DoubleType, true, -2), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, DoubleType, true, -2))), stockBatchQuantities#143789, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, TimestampType, false, -3), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, TimestampType, false, -3))), stockBatchShelfLifeEndDatetimes#143790, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4), createexternalrow(invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandChannel.toString()), invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandStream.toString()), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).considerVariance)), mapobjects(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5), if (isnull(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandStartDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandVariance)), StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandTimeBuckets, Some(class scala.collection.mutable.ArraySeq)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeStart)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeEnd)), mapobjects(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6), if (isnull(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandVariance)), StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTime, Some(class scala.collection.mutable.ArraySeq)), StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)), rawDemands#143413, Some(class scala.collection.mutable.ArraySeq)), StructField(product,StringType,true), StructField(plant,StringType,true), StructField(orderDateTime,TimestampType,true), StructField(productPlant,StructType(StructField(systemId,StringType,true),StructField(internalUUID,StringType,true),StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(blockedForReplenishmentStartingFrom,DateType,true),StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true),StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)),true), StructField(timezonecode,StringType,true), StructField(outboundUnit,DoubleType,true), StructField(stockDatetime,TimestampType,true), StructField(stockBatchQuantities,ArrayType(DoubleType,true),false), StructField(stockBatchShelfLifeEndDatetimes,ArrayType(TimestampType,false),false), StructField(rawDemands,ArrayType(StructType(StructField(demandChannel,StringType,true),StructField(demandStream,StringType,true),StructField(considerVariance,BooleanType,true),StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true),StructField(demandPointInTimeStart,TimestampType,true),StructField(demandPointInTimeEnd,TimestampType,true),StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)),false),false)), obj#143452: org.apache.spark.sql.Row
(751) MapElements
Input [1]: [obj#143452]
Arguments: com.sap.s4hana.eureka.business.crporderquantitykpiservice.core.business.datasnapshot.DemandTransformatorImpl$$Lambda/0x00000008024b0d88@2d5ca6d1, obj#143457: org.apache.spark.sql.Row
(752) SerializeFromObject
Input [1]: [obj#143457]
Arguments: [if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(0))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, product), StringType, ObjectType(class java.lang.String)))) AS product#143462, if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(1))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, plant), StringType, ObjectType(class java.lang.String)))) AS plant#143463]
(753) Exchange
Input [2]: [product#143462, plant#143463]
Arguments: hashpartitioning(coalesce(plant#143463, ), isnull(plant#143463), coalesce(product#143462, ), isnull(product#143462), 25), ENSURE_REQUIREMENTS, [plan_id=217440]
(754) Sort
Input [2]: [product#143462, plant#143463]
Arguments: [coalesce(plant#143463, ) ASC NULLS FIRST, isnull(plant#143463) ASC NULLS FIRST, coalesce(product#143462, ) ASC NULLS FIRST, isnull(product#143462) ASC NULLS FIRST], false, 0
(755) SortMergeJoin
Left keys [4]: [coalesce(plant#142981, ), isnull(plant#142981), coalesce(product#142980, ), isnull(product#142980)]
Right keys [4]: [coalesce(plant#143463, ), isnull(plant#143463), coalesce(product#143462, ), isnull(product#143462)]
Join type: LeftOuter
Join condition: None
(756) Project
Output [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Input [6]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384, product#143462, plant#143463]
(757) Exchange
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: hashpartitioning(plant#142981, 25), ENSURE_REQUIREMENTS, [plan_id=217470]
(758) Sort
Input [4]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384]
Arguments: [plant#142981 ASC NULLS FIRST], false, 0
(759) Scan parquet
Output [2]: [marketUnit#143855, distributionCenters#143861]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(760) Filter
Input [2]: [marketUnit#143855, distributionCenters#143861]
Condition : (((marketUnit#143855 <=> Daily_grocery) AND (size(distributionCenters#143861, true) > 0)) AND isnotnull(distributionCenters#143861))
(761) Project
Output [1]: [distributionCenters#143861]
Input [2]: [marketUnit#143855, distributionCenters#143861]
(762) Exchange
Input [1]: [distributionCenters#143861]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216891]
(763) Generate
Input [1]: [distributionCenters#143861]
Arguments: explode(distributionCenters#143861), false, [distributionCenters#143877]
(764) Filter
Input [1]: [distributionCenters#143877]
Condition : (((size(distributionCenters#143877.storageLocations.internalRefUUID, true) > 0) AND isnotnull(distributionCenters#143877.storageLocations.internalRefUUID)) AND isnotnull(distributionCenters#143877.internalRefUUID))
(765) Project
Output [2]: [distributionCenters#143877.internalRefUUID AS _extract_internalRefUUID#143980, distributionCenters#143877.storageLocations.internalRefUUID AS _extract_internalRefUUID#144165]
Input [1]: [distributionCenters#143877]
(766) Generate
Input [2]: [_extract_internalRefUUID#143980, _extract_internalRefUUID#144165]
Arguments: explode(_extract_internalRefUUID#144165), [_extract_internalRefUUID#143980], false, [storageLocations#143878]
(767) Filter
Input [2]: [_extract_internalRefUUID#143980, storageLocations#143878]
Condition : isnotnull(storageLocations#143878)
(768) Project
Output [2]: [_extract_internalRefUUID#143980 AS plantUuid#143880, storageLocations#143878 AS storageLocUuid#143882]
Input [2]: [_extract_internalRefUUID#143980, storageLocations#143878]
(769) BroadcastExchange
Input [2]: [plantUuid#143880, storageLocUuid#143882]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217465]
(770) Scan parquet
Output [2]: [product#143883, plant#143884]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002108/shardId=0_1_10000002108]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(771) Filter
Input [2]: [product#143883, plant#143884]
Condition : isnotnull(plant#143884)
(772) Exchange
Input [2]: [product#143883, plant#143884]
Arguments: hashpartitioning(product#143883, plant#143884, 25), REPARTITION_BY_NUM, [plan_id=217119]
(773) Scan parquet
Output [4]: [plant#143889, product#143890, productionAspect#143892, systemId#143887]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(774) Project
Output [4]: [plant#143889.internalRefUUID AS _extract_internalRefUUID#144166, product#143890.internalRefUUID AS _extract_internalRefUUID#144167, productionAspect#143892.productMovementPlants.unitOfIssue.code AS _extract_code#144168, productionAspect#143892.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#144169]
Input [4]: [plant#143889, product#143890, productionAspect#143892, systemId#143887]
(775) Exchange
Input [4]: [_extract_internalRefUUID#144166, _extract_internalRefUUID#144167, _extract_code#144168, _extract_internalRefUUID#144169]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216904]
(776) Project
Output [4]: [_extract_code#144168, _extract_internalRefUUID#144169, _extract_internalRefUUID#144167, _extract_internalRefUUID#144166]
Input [4]: [_extract_internalRefUUID#144166, _extract_internalRefUUID#144167, _extract_code#144168, _extract_internalRefUUID#144169]
(777) BroadcastExchange
Input [4]: [_extract_code#144168, _extract_internalRefUUID#144169, _extract_internalRefUUID#144167, _extract_internalRefUUID#144166]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=217445]
(778) BroadcastHashJoin
Left keys [4]: [coalesce(product#143883, ), isnull(product#143883), coalesce(plant#143884, ), isnull(plant#143884)]
Right keys [4]: [coalesce(_extract_internalRefUUID#144167, ), isnull(_extract_internalRefUUID#144167), coalesce(_extract_internalRefUUID#144166, ), isnull(_extract_internalRefUUID#144166)]
Join type: LeftOuter
Join condition: None
(779) Project
Output [4]: [product#143883, plant#143884, _extract_code#144168, _extract_internalRefUUID#144169]
Input [6]: [product#143883, plant#143884, _extract_code#144168, _extract_internalRefUUID#144169, _extract_internalRefUUID#144167, _extract_internalRefUUID#144166]
(780) Scan parquet
Output [2]: [internalUUID#143897, systemId#143896]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(781) Project
Output [1]: [internalUUID#143897]
Input [2]: [internalUUID#143897, systemId#143896]
(782) Exchange
Input [1]: [internalUUID#143897]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216910]
(783) BroadcastExchange
Input [1]: [internalUUID#143897]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=217449]
(784) BroadcastHashJoin
Left keys [2]: [coalesce(plant#143884, ), isnull(plant#143884)]
Right keys [2]: [coalesce(internalUUID#143897, ), isnull(internalUUID#143897)]
Join type: LeftOuter
Join condition: None
(785) Project
Output [4]: [product#143883, plant#143884, _extract_code#144168, _extract_internalRefUUID#144169]
Input [5]: [product#143883, plant#143884, _extract_code#144168, _extract_internalRefUUID#144169, internalUUID#143897]
(786) Scan parquet
Output [4]: [internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917, systemId#143910]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(787) Filter
Input [4]: [internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917, systemId#143910]
Condition : isnotnull(internalUUID#143911)
(788) Project
Output [3]: [internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917]
Input [4]: [internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917, systemId#143910]
(789) Exchange
Input [3]: [internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216916]
(790) BroadcastExchange
Input [3]: [internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=217453]
(791) BroadcastHashJoin
Left keys [1]: [product#143883]
Right keys [1]: [internalUUID#143911]
Join type: LeftOuter
Join condition: None
(792) Project
Output [2]: [plant#143884, filter(unitOfMeasureConversions#143917, lambdafunction((((lambda u#143027.measurementUnit1.code = _extract_code#144168) AND (lambda u#143027.measurementUnit1.internalRefUUID = _extract_internalRefUUID#144169)) AND ((lambda u#143027.measurementUnit2.code = baseUnitOfMeasure#143915.code) AND (lambda u#143027.measurementUnit2.internalRefUUID = baseUnitOfMeasure#143915.internalRefUUID))), lambda u#143027, false)) AS filteredUnitOfMeasureConversions#143922]
Input [7]: [product#143883, plant#143884, _extract_code#144168, _extract_internalRefUUID#144169, internalUUID#143911, baseUnitOfMeasure#143915, unitOfMeasureConversions#143917]
(793) Generate
Input [2]: [plant#143884, filteredUnitOfMeasureConversions#143922]
Arguments: explode(filteredUnitOfMeasureConversions#143922.quantityNumerator), [plant#143884, filteredUnitOfMeasureConversions#143922], true, [quantityNumerator#143923]
(794) Project
Output [2]: [plant#143884, filteredUnitOfMeasureConversions#143922.quantityDenominator AS _extract_quantityDenominator#143982]
Input [3]: [plant#143884, filteredUnitOfMeasureConversions#143922, quantityNumerator#143923]
(795) Generate
Input [2]: [plant#143884, _extract_quantityDenominator#143982]
Arguments: explode(_extract_quantityDenominator#143982), [plant#143884], true, [quantityDenominator#143924]
(796) Project
Output [1]: [plant#143884]
Input [2]: [plant#143884, quantityDenominator#143924]
(797) HashAggregate
Input [1]: [plant#143884]
Keys [1]: [plant#143884]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143884]
(798) Exchange
Input [1]: [plant#143884]
Arguments: hashpartitioning(plant#143884, 25), ENSURE_REQUIREMENTS, [plan_id=217462]
(799) HashAggregate
Input [1]: [plant#143884]
Keys [1]: [plant#143884]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#143884]
(800) BroadcastHashJoin
Left keys [1]: [plantUuid#143880]
Right keys [1]: [plant#143884]
Join type: Inner
Join condition: None
(801) Project
Output [2]: [plantUuid#143880 AS plant#143854, storageLocUuid#143882]
Input [3]: [plantUuid#143880, storageLocUuid#143882, plant#143884]
(802) Sort
Input [2]: [plant#143854, storageLocUuid#143882]
Arguments: [plant#143854 ASC NULLS FIRST], false, 0
(803) SortMergeJoin
Left keys [1]: [plant#142981]
Right keys [1]: [plant#143854]
Join type: Inner
Join condition: None
(804) Project
Output [5]: [plant#142981, product#142980, timezonecode#143012, stockDatetime#143384, storageLocUuid#143882]
Input [6]: [product#142980, plant#142981, timezonecode#143012, stockDatetime#143384, plant#143854, storageLocUuid#143882]
(805) Filter
Input [6]: [internalUUID#143816, id#143817, supplier#143819, isMarkedForDeletion#143822, items#143823, systemId#143815]
Condition : (((isnotnull(isMarkedForDeletion#143822) AND NOT isMarkedForDeletion#143822) AND (size(items#143823, true) > 0)) AND isnotnull(items#143823))
(806) Project
Output [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Input [6]: [internalUUID#143816, id#143817, supplier#143819, isMarkedForDeletion#143822, items#143823, systemId#143815]
(807) Exchange
Input [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=216932]
(808) Generate
Input [4]: [internalUUID#143816, id#143817, supplier#143819, items#143823]
Arguments: explode(items#143823), [internalUUID#143816, id#143817, supplier#143819], false, [item#143827]
(809) Filter
Input [4]: [internalUUID#143816, id#143817, supplier#143819, item#143827]
Condition : ((((isnotnull(item#143827.deletionIndicator) AND isnotnull(item#143827.completeIndicator)) AND isnotnull(item#143827.returnIndicator)) AND ((NOT item#143827.deletionIndicator AND NOT item#143827.completeIndicator) AND NOT item#143827.returnIndicator)) AND ((isnotnull(item#143827.product.internalRefUUID) AND isnotnull(item#143827.plant.internalRefUUID)) AND isnotnull(item#143827.storageLocation.internalRefUUID)))
(810) Project
Output [14]: [id#143817, internalUUID#143816, supplier#143819.internalRefUUID AS supplierUuid#143824, item#143827.plant.internalRefUUID AS plant#143832, item#143827.product.internalRefUUID AS product#143833, item#143827.plant.id AS externalPlantId#143834, item#143827.product.id AS externalProductId#143835, item#143827.storageLocation.internalRefUUID AS storageLocUuid#143836, item#143827.quantityUnit.code AS quantityUom#143837, item#143827.quantityUnit.internalRefUUID AS quantityUomUuid#143838, item#143827.quantityNumerator AS quantityNumerator#143839, item#143827.quantityDenominator AS quantityDenominator#143840, item#143827.scheduleLines AS scheduleLines#143841, item#143827.confirmations AS confirmations#143842]
Input [4]: [internalUUID#143816, id#143817, supplier#143819, item#143827]
(811) BroadcastExchange
Input [14]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842]
Arguments: HashedRelationBroadcastMode(List(input[4, string, true], input[3, string, true], input[7, string, true]),false), [plan_id=217476]
(812) BroadcastHashJoin
Left keys [3]: [product#142980, plant#142981, storageLocUuid#143882]
Right keys [3]: [product#143833, plant#143832, storageLocUuid#143836]
Join type: Inner
Join condition: None
(813) Project
Output [17]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842, stockDatetime#143384, timezonecode#143012, filter(confirmations#143842, lambdafunction(((NOT lambda x_29#143933.deletionIndicator AND lambda x_29#143933.isRelevantForConfirmation) AND isnotnull(lambda x_29#143933.confirmedDeliveryAt)), lambda x_29#143933, false)) AS validConfirmations#143932]
Input [19]: [plant#142981, product#142980, timezonecode#143012, stockDatetime#143384, storageLocUuid#143882, id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842]
(814) AdaptiveSparkPlan
Output [17]: [id#143817, internalUUID#143816, supplierUuid#143824, plant#143832, product#143833, externalPlantId#143834, externalProductId#143835, storageLocUuid#143836, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, confirmations#143842, stockDatetime#143384, timezonecode#143012, validConfirmations#143932]
Arguments: isFinalPlan=true
(815) TableCacheQueryStage
Output [10]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityDenominator#143840, quantityNumerator#143839, quantityUom#143837, quantityUomUuid#143838, scheduleLines#143841, validConfirmations#143932]
Arguments: 0
(816) Project [codegen id : 1]
Output [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, size(validConfirmations#143932, false) AS _w0#145380]
Input [10]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityDenominator#143840, quantityNumerator#143839, quantityUom#143837, quantityUomUuid#143838, scheduleLines#143841, validConfirmations#143932]
(817) Exchange
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: hashpartitioning(internalUUID#143816, 25), ENSURE_REQUIREMENTS, [plan_id=264227]
(818) ShuffleQueryStage
Output [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: 1
(819) AQEShuffleRead
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: coalesced
(820) Sort [codegen id : 2]
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: [internalUUID#143816 ASC NULLS FIRST], false, 0
(821) Window
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: [sum(_w0#145380) windowspecdefinition(internalUUID#143816, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#145381L], [internalUUID#143816]
(822) Project
Output [3]: [plant#143832, product#143833, filter(transform(CASE WHEN (_we0#145381L > 0) THEN transform(validConfirmations#143932, lambdafunction(struct(openQty, greatest((lambda x_35#145376.confirmedQuantity - coalesce(lambda x_35#145376.mrpRelevantQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_35#145376.confirmedDeliveryAt), lambda x_35#145376, false)) ELSE transform(scheduleLines#143841, lambdafunction(struct(openQty, greatest((lambda x_34#145377.quantity - coalesce(lambda x_34#145377.receivedQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_34#145377.deliveryAt), lambda x_34#145377, false)) END, lambdafunction(struct(documentId, id#143817, documentUuid, internalUUID#143816, openQty, lambda x_36#145378.openQty, openQtyInBaseUnit, ((lambda x_36#145378.openQty * quantityNumerator#143839) / quantityDenominator#143840), qtyUom, quantityUom#143837, qtyUomUuid, quantityUomUuid#143838, availabilityDatetime, lambda x_36#145378.availabilityDatetime, shelfLifeEndDatetime, 2199-01-01 00:00:00), lambda x_36#145378, false)), lambdafunction(NOT (lambda x_37#145379.openQty = 0.0), lambda x_37#145379, false)) AS openGoodsMovements#145374]
Input [12]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380, _we0#145381L]
(823) ObjectHashAggregate
Input [3]: [plant#143832, product#143833, openGoodsMovements#145374]
Keys [2]: [product#143833, plant#143832]
Functions [1]: [partial_collect_list(openGoodsMovements#145374, 0, 0)]
Aggregate Attributes [1]: [buf#145657]
Results [3]: [product#143833, plant#143832, buf#145658]
(824) Exchange
Input [3]: [product#143833, plant#143832, buf#145658]
Arguments: hashpartitioning(product#143833, plant#143832, 25), ENSURE_REQUIREMENTS, [plan_id=264286]
(825) ShuffleQueryStage
Output [3]: [product#143833, plant#143832, buf#145658]
Arguments: 2
(826) ObjectHashAggregate
Input [3]: [product#143833, plant#143832, buf#145658]
Keys [2]: [product#143833, plant#143832]
Functions [1]: [collect_list(openGoodsMovements#145374, 0, 0)]
Aggregate Attributes [1]: [collect_list(openGoodsMovements#145374, 0, 0)#145401]
Results [3]: [product#143833, plant#143832, flatten(collect_list(openGoodsMovements#145374, 0, 0)#145401) AS openGoodsMovements#145382]
(827) Filter [codegen id : 3]
Input [3]: [product#143833, plant#143832, openGoodsMovements#145382]
Condition : (size(openGoodsMovements#145382, false) > 0)
(828) ResultQueryStage
Output [3]: [product#143833, plant#143832, openGoodsMovements#145382]
Arguments: 3
(829) Project
Output [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, size(validConfirmations#143932, false) AS _w0#145380]
Input [10]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityDenominator#143840, quantityNumerator#143839, quantityUom#143837, quantityUomUuid#143838, scheduleLines#143841, validConfirmations#143932]
(830) Exchange
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: hashpartitioning(internalUUID#143816, 25), ENSURE_REQUIREMENTS, [plan_id=217504]
(831) Sort
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: [internalUUID#143816 ASC NULLS FIRST], false, 0
(832) Window
Input [11]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380]
Arguments: [sum(_w0#145380) windowspecdefinition(internalUUID#143816, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#145381L], [internalUUID#143816]
(833) Project
Output [3]: [plant#143832, product#143833, filter(transform(CASE WHEN (_we0#145381L > 0) THEN transform(validConfirmations#143932, lambdafunction(struct(openQty, greatest((lambda x_35#145376.confirmedQuantity - coalesce(lambda x_35#145376.mrpRelevantQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_35#145376.confirmedDeliveryAt), lambda x_35#145376, false)) ELSE transform(scheduleLines#143841, lambdafunction(struct(openQty, greatest((lambda x_34#145377.quantity - coalesce(lambda x_34#145377.receivedQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_34#145377.deliveryAt), lambda x_34#145377, false)) END, lambdafunction(struct(documentId, id#143817, documentUuid, internalUUID#143816, openQty, lambda x_36#145378.openQty, openQtyInBaseUnit, ((lambda x_36#145378.openQty * quantityNumerator#143839) / quantityDenominator#143840), qtyUom, quantityUom#143837, qtyUomUuid, quantityUomUuid#143838, availabilityDatetime, lambda x_36#145378.availabilityDatetime, shelfLifeEndDatetime, 2199-01-01 00:00:00), lambda x_36#145378, false)), lambdafunction(NOT (lambda x_37#145379.openQty = 0.0), lambda x_37#145379, false)) AS openGoodsMovements#145374]
Input [12]: [id#143817, internalUUID#143816, plant#143832, product#143833, quantityUom#143837, quantityUomUuid#143838, quantityNumerator#143839, quantityDenominator#143840, scheduleLines#143841, validConfirmations#143932, _w0#145380, _we0#145381L]
(834) ObjectHashAggregate
Input [3]: [plant#143832, product#143833, openGoodsMovements#145374]
Keys [2]: [product#143833, plant#143832]
Functions [1]: [partial_collect_list(openGoodsMovements#145374, 0, 0)]
Aggregate Attributes [1]: [buf#145657]
Results [3]: [product#143833, plant#143832, buf#145658]
(835) Exchange
Input [3]: [product#143833, plant#143832, buf#145658]
Arguments: hashpartitioning(product#143833, plant#143832, 25), ENSURE_REQUIREMENTS, [plan_id=217510]
(836) ObjectHashAggregate
Input [3]: [product#143833, plant#143832, buf#145658]
Keys [2]: [product#143833, plant#143832]
Functions [1]: [collect_list(openGoodsMovements#145374, 0, 0)]
Aggregate Attributes [1]: [collect_list(openGoodsMovements#145374, 0, 0)#145401]
Results [3]: [product#143833, plant#143832, flatten(collect_list(openGoodsMovements#145374, 0, 0)#145401) AS openGoodsMovements#145382]
(837) Filter
Input [3]: [product#143833, plant#143832, openGoodsMovements#145382]
Condition : (size(openGoodsMovements#145382, false) > 0)
(838) AdaptiveSparkPlan
Output [3]: [product#143833, plant#143832, openGoodsMovements#145382]
Arguments: isFinalPlan=true
(839) TableCacheQueryStage
Output [3]: [openGoodsMovements#145382, plant#143832, product#143833]
Arguments: 0
(840) Filter
Input [3]: [openGoodsMovements#145382, plant#143832, product#143833]
Condition : (size(transform(openGoodsMovements#145382, lambdafunction(struct(documentType, PO, documentId, lambda x_42#153457.documentId, documentUuid, lambda x_42#153457.documentUuid, openQty, lambda x_42#153457.openQty, qtyUomUuid, lambda x_42#153457.qtyUomUuid, qtyUom, lambda x_42#153457.qtyUom, plannedDeliveryDatetime, lambda x_42#153457.availabilityDatetime), lambda x_42#153457, false)), false) > 0)
(841) Project
Output [4]: [product#143833, plant#143832 AS location#153453, transform(openGoodsMovements#145382, lambdafunction(struct(documentType, PO, documentId, lambda x_42#153457.documentId, documentUuid, lambda x_42#153457.documentUuid, openQty, lambda x_42#153457.openQty, qtyUomUuid, lambda x_42#153457.qtyUomUuid, qtyUom, lambda x_42#153457.qtyUom, plannedDeliveryDatetime, lambda x_42#153457.availabilityDatetime), lambda x_42#153457, false)) AS documents#153456, 10000002108 AS replenishmentRunId#153458L]
Input [3]: [openGoodsMovements#145382, plant#143832, product#143833]
(842) WriteFiles
Input [4]: [product#143833, location#153453, documents#153456, replenishmentRunId#153458L]
(843) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002108/0_1_10000002108, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002108/0_1_10000002108], Append, [product, location, documents, replenishmentRunId]
(844) ResultQueryStage
Output: []
Arguments: 1
(845) Filter
Input [3]: [openGoodsMovements#145382, plant#143832, product#143833]
Condition : (size(transform(openGoodsMovements#145382, lambdafunction(struct(documentType, PO, documentId, lambda x_42#153457.documentId, documentUuid, lambda x_42#153457.documentUuid, openQty, lambda x_42#153457.openQty, qtyUomUuid, lambda x_42#153457.qtyUomUuid, qtyUom, lambda x_42#153457.qtyUom, plannedDeliveryDatetime, lambda x_42#153457.availabilityDatetime), lambda x_42#153457, false)), false) > 0)
(846) Project
Output [4]: [product#143833, plant#143832 AS location#153453, transform(openGoodsMovements#145382, lambdafunction(struct(documentType, PO, documentId, lambda x_42#153457.documentId, documentUuid, lambda x_42#153457.documentUuid, openQty, lambda x_42#153457.openQty, qtyUomUuid, lambda x_42#153457.qtyUomUuid, qtyUom, lambda x_42#153457.qtyUom, plannedDeliveryDatetime, lambda x_42#153457.availabilityDatetime), lambda x_42#153457, false)) AS documents#153456, 10000002108 AS replenishmentRunId#153458L]
Input [3]: [openGoodsMovements#145382, plant#143832, product#143833]
(847) WriteFiles
Input [4]: [product#143833, location#153453, documents#153456, replenishmentRunId#153458L]
(848) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002108/0_1_10000002108, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002108/0_1_10000002108], Append, [product, location, documents, replenishmentRunId]
(849) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true
| Name | Value |
|---|---|
| spark.activity.hdlfsEnabled | true |
| spark.activity.isolateSession | true |
| spark.databricks.delta.lastCommitVersionInSession | 0 |
| spark.sql.parquet.fieldId.read.enabled | true |
| spark.sql.parquet.fieldId.write.enabled | true |
| temporal.activity.id | 6df44603-5e2c-38fa-85f2-dbda359fe891 |
| temporal.activity.type | BufferDataSnapShot |