== Physical Plan ==
AdaptiveSparkPlan (837)
+- == Final Plan ==
ResultQueryStage (832), Statistics(sizeInBytes=8.0 EiB)
+- Execute InsertIntoHadoopFsRelationCommand (831)
+- WriteFiles (830)
+- Project (829)
+- Filter (828)
+- TableCacheQueryStage (827), Statistics(sizeInBytes=0.0 B, rowCount=0)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- AdaptiveSparkPlan (826)
+- == Final Plan ==
ResultQueryStage (4)
+- EmptyRelation (3)
+- Filter (unknown)
+- EmptyRelation (unknown)
+- == Initial Plan ==
Filter (825)
+- ObjectHashAggregate (824)
+- Exchange (823)
+- ObjectHashAggregate (822)
+- Project (821)
+- Window (820)
+- Sort (819)
+- Exchange (818)
+- Project (817)
+- InMemoryTableScan (5)
+- InMemoryRelation (6)
+- AdaptiveSparkPlan (816)
+- == Final Plan ==
ResultQueryStage (323)
+- Project (322)
+- * BroadcastHashJoin Inner BuildRight (321)
:- * Project (309)
: +- * BroadcastHashJoin Inner BuildLeft (308)
: :- BroadcastQueryStage (292), Statistics(sizeInBytes=2.0 MiB, rowCount=205)
: : +- BroadcastExchange (291)
: : +- AQEShuffleRead (290)
: : +- ShuffleQueryStage (289), Statistics(sizeInBytes=24.0 KiB, rowCount=205)
: : +- Exchange (288)
: : +- * Project (287)
: : +- * BroadcastHashJoin LeftOuter BuildRight (286)
: : :- AQEShuffleRead (180)
: : : +- ShuffleQueryStage (179), Statistics(sizeInBytes=24.0 KiB, rowCount=205)
: : : +- Exchange (178)
: : : +- * Project (177)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (176)
: : : :- * Project (44)
: : : : +- * Generate (43)
: : : : +- * Project (42)
: : : : +- * Generate (41)
: : : : +- Project (40)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (39)
: : : : :- * Project (30)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (29)
: : : : : :- * Project (21)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (20)
: : : : : : :- ShuffleQueryStage (11), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- Exchange (10)
: : : : : : : +- * Filter (9)
: : : : : : : +- * ColumnarToRow (8)
: : : : : : : +- Scan parquet (7)
: : : : : : +- BroadcastQueryStage (19), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (18)
: : : : : : +- * Project (17)
: : : : : : +- ShuffleQueryStage (16), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (15)
: : : : : : +- * Project (14)
: : : : : : +- * ColumnarToRow (13)
: : : : : : +- Scan parquet (12)
: : : : : +- BroadcastQueryStage (28), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (27)
: : : : : +- ShuffleQueryStage (26), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (25)
: : : : : +- * Project (24)
: : : : : +- * ColumnarToRow (23)
: : : : : +- Scan parquet (22)
: : : : +- BroadcastQueryStage (38), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (37)
: : : : +- ShuffleQueryStage (36), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (35)
: : : : +- * Project (34)
: : : : +- * Filter (33)
: : : : +- * ColumnarToRow (32)
: : : : +- Scan parquet (31)
: : : +- BroadcastQueryStage (175), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : +- BroadcastExchange (174)
: : : +- * HashAggregate (173)
: : : +- AQEShuffleRead (172)
: : : +- ShuffleQueryStage (171), Statistics(sizeInBytes=4.1 KiB, rowCount=44)
: : : +- Exchange (170)
: : : +- * HashAggregate (169)
: : : +- * Project (168)
: : : +- * BroadcastHashJoin Inner BuildLeft (167)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : : +- BroadcastExchange (138)
: : : : +- AQEShuffleRead (137)
: : : : +- ShuffleQueryStage (136), Statistics(sizeInBytes=4.8 KiB, rowCount=44)
: : : : +- Exchange (135)
: : : : +- * Project (134)
: : : : +- * BroadcastHashJoin Inner BuildLeft (133)
: : : : :- BroadcastQueryStage (111), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (110)
: : : : : +- AQEShuffleRead (109)
: : : : : +- ShuffleQueryStage (108), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (107)
: : : : : +- * Project (106)
: : : : : +- * Filter (105)
: : : : : +- * Generate (104)
: : : : : +- * Filter (103)
: : : : : +- ObjectHashAggregate (102)
: : : : : +- ObjectHashAggregate (101)
: : : : : +- Generate (100)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (99)
: : : : : :- BroadcastQueryStage (52), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (51)
: : : : : : +- ShuffleQueryStage (50), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (49)
: : : : : : +- * Project (48)
: : : : : : +- * Filter (47)
: : : : : : +- * ColumnarToRow (46)
: : : : : : +- Scan parquet (45)
: : : : : +- * Project (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (65), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (64)
: : : : : : +- * Project (63)
: : : : : : +- * Generate (62)
: : : : : : +- * Project (61)
: : : : : : +- * Filter (60)
: : : : : : +- * Generate (59)
: : : : : : +- ShuffleQueryStage (58), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (57)
: : : : : : +- * Project (56)
: : : : : : +- * Filter (55)
: : : : : : +- * ColumnarToRow (54)
: : : : : : +- Scan parquet (53)
: : : : : +- * HashAggregate (96)
: : : : : +- AQEShuffleRead (95)
: : : : : +- ShuffleQueryStage (94), Statistics(sizeInBytes=7.2 KiB, rowCount=153)
: : : : : +- Exchange (93)
: : : : : +- * HashAggregate (92)
: : : : : +- * Project (91)
: : : : : +- * Generate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- Project (87)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (86)
: : : : : :- * Project (83)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (82)
: : : : : : :- * Project (74)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (73)
: : : : : : : :- ShuffleQueryStage (70), Statistics(sizeInBytes=17.6 KiB, rowCount=205)
: : : : : : : : +- Exchange (69)
: : : : : : : : +- * Filter (68)
: : : : : : : : +- * ColumnarToRow (67)
: : : : : : : : +- Scan parquet (66)
: : : : : : : +- BroadcastQueryStage (72), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (71)
: : : : : : +- BroadcastQueryStage (81), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (80)
: : : : : : +- ShuffleQueryStage (79), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (78)
: : : : : : +- * Project (77)
: : : : : : +- * ColumnarToRow (76)
: : : : : : +- Scan parquet (75)
: : : : : +- BroadcastQueryStage (85), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (84)
: : : : +- * Project (132)
: : : : +- * Generate (131)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- Project (128)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (127)
: : : : :- * Project (124)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (123)
: : : : : :- * Project (120)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (119)
: : : : : : :- ShuffleQueryStage (116), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- Exchange (115)
: : : : : : : +- * Filter (114)
: : : : : : : +- * ColumnarToRow (113)
: : : : : : : +- Scan parquet (112)
: : : : : : +- BroadcastQueryStage (118), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (117)
: : : : : +- BroadcastQueryStage (122), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (121)
: : : : +- BroadcastQueryStage (126), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (125)
: : : +- AQEShuffleRead (166)
: : : +- ShuffleQueryStage (165), Statistics(sizeInBytes=4.8 KiB, rowCount=44)
: : : +- Exchange (164)
: : : +- * HashAggregate (163)
: : : +- * HashAggregate (162)
: : : +- * Project (161)
: : : +- * BroadcastHashJoin Inner BuildLeft (160)
: : : :- BroadcastQueryStage (141), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (140)
: : : +- * Project (159)
: : : +- * Generate (158)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- Project (155)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (154)
: : : :- * Project (151)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (150)
: : : : :- * Project (147)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (146)
: : : : : :- ShuffleQueryStage (143), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : +- ReusedExchange (142)
: : : : : +- BroadcastQueryStage (145), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (144)
: : : : +- BroadcastQueryStage (149), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (148)
: : : +- BroadcastQueryStage (153), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (152)
: : +- BroadcastQueryStage (285), Statistics(sizeInBytes=2.0 MiB, rowCount=205)
: : +- BroadcastExchange (284)
: : +- AQEShuffleRead (283)
: : +- ShuffleQueryStage (282), Statistics(sizeInBytes=17.6 KiB, rowCount=205)
: : +- Exchange (281)
: : +- * SerializeFromObject (280)
: : +- * MapElements (279)
: : +- * DeserializeToObject (278)
: : +- ObjectHashAggregate (277)
: : +- ObjectHashAggregate (276)
: : +- * Project (275)
: : +- * BroadcastHashJoin LeftOuter BuildRight (274)
: : :- * Project (268)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (267)
: : : :- * Project (206)
: : : : +- * Project (205)
: : : : +- * Generate (204)
: : : : +- * Generate (203)
: : : : +- * Project (202)
: : : : +- Project (201)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (200)
: : : : :- * Project (197)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (196)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (193)
: : : : : : :- ShuffleQueryStage (184), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- Exchange (183)
: : : : : : : +- * ColumnarToRow (182)
: : : : : : : +- Scan parquet (181)
: : : : : : +- BroadcastQueryStage (192), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (191)
: : : : : : +- * Project (190)
: : : : : : +- ShuffleQueryStage (189), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (188)
: : : : : : +- * Project (187)
: : : : : : +- * ColumnarToRow (186)
: : : : : : +- Scan parquet (185)
: : : : : +- BroadcastQueryStage (195), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (194)
: : : : +- BroadcastQueryStage (199), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (198)
: : : +- BroadcastQueryStage (266), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : +- BroadcastExchange (265)
: : : +- * HashAggregate (264)
: : : +- AQEShuffleRead (263)
: : : +- ShuffleQueryStage (262), Statistics(sizeInBytes=4.5 KiB, rowCount=44)
: : : +- Exchange (261)
: : : +- * HashAggregate (260)
: : : +- * Project (259)
: : : +- * BroadcastHashJoin Inner BuildLeft (258)
: : : :- BroadcastQueryStage (254), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : : +- BroadcastExchange (253)
: : : : +- AQEShuffleRead (252)
: : : : +- ShuffleQueryStage (251), Statistics(sizeInBytes=5.2 KiB, rowCount=44)
: : : : +- Exchange (250)
: : : : +- * Project (249)
: : : : +- * BroadcastHashJoin Inner BuildLeft (248)
: : : : :- BroadcastQueryStage (229), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (228)
: : : : : +- AQEShuffleRead (227)
: : : : : +- ShuffleQueryStage (226), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (225)
: : : : : +- * Project (224)
: : : : : +- * Filter (223)
: : : : : +- * Generate (222)
: : : : : +- * Filter (221)
: : : : : +- ObjectHashAggregate (220)
: : : : : +- ObjectHashAggregate (219)
: : : : : +- Generate (218)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (217)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (207)
: : : : : +- * Project (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (210), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (209)
: : : : : +- * HashAggregate (214)
: : : : : +- AQEShuffleRead (213)
: : : : : +- ShuffleQueryStage (212), Statistics(sizeInBytes=7.2 KiB, rowCount=153)
: : : : : +- ReusedExchange (211)
: : : : +- * Project (247)
: : : : +- * Generate (246)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- Project (243)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (242)
: : : : :- * Project (239)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (238)
: : : : : :- * Project (235)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (234)
: : : : : : :- ShuffleQueryStage (231), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- ReusedExchange (230)
: : : : : : +- BroadcastQueryStage (233), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (232)
: : : : : +- BroadcastQueryStage (237), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (236)
: : : : +- BroadcastQueryStage (241), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (240)
: : : +- AQEShuffleRead (257)
: : : +- ShuffleQueryStage (256), Statistics(sizeInBytes=4.8 KiB, rowCount=44)
: : : +- ReusedExchange (255)
: : +- BroadcastQueryStage (273), Statistics(sizeInBytes=2.0 MiB, rowCount=205)
: : +- BroadcastExchange (272)
: : +- * Filter (271)
: : +- * ColumnarToRow (270)
: : +- Scan parquet (269)
: +- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (302), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (301)
: : +- * Project (300)
: : +- * Filter (299)
: : +- * Generate (298)
: : +- * Project (297)
: : +- * Filter (296)
: : +- * Generate (295)
: : +- ShuffleQueryStage (294), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (293)
: +- * HashAggregate (305)
: +- ShuffleQueryStage (304), Statistics(sizeInBytes=7.2 KiB, rowCount=153)
: +- ReusedExchange (303)
+- BroadcastQueryStage (320), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (319)
+- * Project (318)
+- * Filter (317)
+- * Generate (316)
+- ShuffleQueryStage (315), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (314)
+- * Project (313)
+- * Filter (312)
+- * ColumnarToRow (311)
+- Scan parquet (310)
+- == Initial Plan ==
Project (815)
+- BroadcastHashJoin Inner BuildRight (814)
:- Project (806)
: +- SortMergeJoin Inner (805)
: :- Sort (760)
: : +- Exchange (759)
: : +- Project (758)
: : +- SortMergeJoin LeftOuter (757)
: : :- Sort (532)
: : : +- Exchange (531)
: : : +- Project (530)
: : : +- SortMergeJoin LeftOuter (529)
: : : :- Sort (347)
: : : : +- Project (346)
: : : : +- Generate (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (341)
: : : : :- Project (336)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (335)
: : : : : :- Project (331)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (330)
: : : : : : :- Exchange (325)
: : : : : : : +- Filter (324)
: : : : : : : +- Scan parquet (7)
: : : : : : +- BroadcastExchange (329)
: : : : : : +- Project (328)
: : : : : : +- Exchange (327)
: : : : : : +- Project (326)
: : : : : : +- Scan parquet (12)
: : : : : +- BroadcastExchange (334)
: : : : : +- Exchange (333)
: : : : : +- Project (332)
: : : : : +- Scan parquet (22)
: : : : +- BroadcastExchange (340)
: : : : +- Exchange (339)
: : : : +- Project (338)
: : : : +- Filter (337)
: : : : +- Scan parquet (31)
: : : +- Sort (528)
: : : +- HashAggregate (527)
: : : +- Exchange (526)
: : : +- HashAggregate (525)
: : : +- Project (524)
: : : +- SortMergeJoin Inner (523)
: : : :- Sort (431)
: : : : +- Exchange (430)
: : : : +- Project (429)
: : : : +- SortMergeJoin Inner (428)
: : : : :- Sort (400)
: : : : : +- Exchange (399)
: : : : : +- Project (398)
: : : : : +- Filter (397)
: : : : : +- Generate (396)
: : : : : +- Filter (395)
: : : : : +- ObjectHashAggregate (394)
: : : : : +- ObjectHashAggregate (393)
: : : : : +- Generate (392)
: : : : : +- BroadcastHashJoin Inner BuildLeft (391)
: : : : : :- BroadcastExchange (351)
: : : : : : +- Exchange (350)
: : : : : : +- Project (349)
: : : : : : +- Filter (348)
: : : : : : +- Scan parquet (45)
: : : : : +- Project (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (360)
: : : : : : +- Project (359)
: : : : : : +- Generate (358)
: : : : : : +- Project (357)
: : : : : : +- Filter (356)
: : : : : : +- Generate (355)
: : : : : : +- Exchange (354)
: : : : : : +- Project (353)
: : : : : : +- Filter (352)
: : : : : : +- Scan parquet (53)
: : : : : +- HashAggregate (388)
: : : : : +- Exchange (387)
: : : : : +- HashAggregate (386)
: : : : : +- Project (385)
: : : : : +- Generate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (380)
: : : : : :- Project (374)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (373)
: : : : : : :- Project (369)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (368)
: : : : : : : :- Exchange (362)
: : : : : : : : +- Filter (361)
: : : : : : : : +- Scan parquet (66)
: : : : : : : +- BroadcastExchange (367)
: : : : : : : +- Project (366)
: : : : : : : +- Exchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Scan parquet (363)
: : : : : : +- BroadcastExchange (372)
: : : : : : +- Exchange (371)
: : : : : : +- Project (370)
: : : : : : +- Scan parquet (75)
: : : : : +- BroadcastExchange (379)
: : : : : +- Exchange (378)
: : : : : +- Project (377)
: : : : : +- Filter (376)
: : : : : +- Scan parquet (375)
: : : : +- Sort (427)
: : : : +- Project (426)
: : : : +- Generate (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (421)
: : : : :- Project (415)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (414)
: : : : : :- Project (409)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (408)
: : : : : : :- Exchange (402)
: : : : : : : +- Filter (401)
: : : : : : : +- Scan parquet (112)
: : : : : : +- BroadcastExchange (407)
: : : : : : +- Project (406)
: : : : : : +- Exchange (405)
: : : : : : +- Project (404)
: : : : : : +- Scan parquet (403)
: : : : : +- BroadcastExchange (413)
: : : : : +- Exchange (412)
: : : : : +- Project (411)
: : : : : +- Scan parquet (410)
: : : : +- BroadcastExchange (420)
: : : : +- Exchange (419)
: : : : +- Project (418)
: : : : +- Filter (417)
: : : : +- Scan parquet (416)
: : : +- Sort (522)
: : : +- Exchange (521)
: : : +- HashAggregate (520)
: : : +- HashAggregate (519)
: : : +- Project (518)
: : : +- SortMergeJoin Inner (517)
: : : :- Sort (488)
: : : : +- Exchange (487)
: : : : +- Project (486)
: : : : +- Filter (485)
: : : : +- Generate (484)
: : : : +- Filter (483)
: : : : +- ObjectHashAggregate (482)
: : : : +- ObjectHashAggregate (481)
: : : : +- Generate (480)
: : : : +- BroadcastHashJoin Inner BuildLeft (479)
: : : : :- BroadcastExchange (436)
: : : : : +- Exchange (435)
: : : : : +- Project (434)
: : : : : +- Filter (433)
: : : : : +- Scan parquet (432)
: : : : +- Project (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (446)
: : : : : +- Project (445)
: : : : : +- Generate (444)
: : : : : +- Project (443)
: : : : : +- Filter (442)
: : : : : +- Generate (441)
: : : : : +- Exchange (440)
: : : : : +- Project (439)
: : : : : +- Filter (438)
: : : : : +- Scan parquet (437)
: : : : +- HashAggregate (476)
: : : : +- Exchange (475)
: : : : +- HashAggregate (474)
: : : : +- Project (473)
: : : : +- Generate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (468)
: : : : :- Project (462)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (461)
: : : : : :- Project (456)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (455)
: : : : : : :- Exchange (449)
: : : : : : : +- Filter (448)
: : : : : : : +- Scan parquet (447)
: : : : : : +- BroadcastExchange (454)
: : : : : : +- Project (453)
: : : : : : +- Exchange (452)
: : : : : : +- Project (451)
: : : : : : +- Scan parquet (450)
: : : : : +- BroadcastExchange (460)
: : : : : +- Exchange (459)
: : : : : +- Project (458)
: : : : : +- Scan parquet (457)
: : : : +- BroadcastExchange (467)
: : : : +- Exchange (466)
: : : : +- Project (465)
: : : : +- Filter (464)
: : : : +- Scan parquet (463)
: : : +- Sort (516)
: : : +- Project (515)
: : : +- Generate (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- BroadcastHashJoin LeftOuter BuildRight (510)
: : : :- Project (504)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (503)
: : : : :- Project (498)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (497)
: : : : : :- Exchange (491)
: : : : : : +- Filter (490)
: : : : : : +- Scan parquet (489)
: : : : : +- BroadcastExchange (496)
: : : : : +- Project (495)
: : : : : +- Exchange (494)
: : : : : +- Project (493)
: : : : : +- Scan parquet (492)
: : : : +- BroadcastExchange (502)
: : : : +- Exchange (501)
: : : : +- Project (500)
: : : : +- Scan parquet (499)
: : : +- BroadcastExchange (509)
: : : +- Exchange (508)
: : : +- Project (507)
: : : +- Filter (506)
: : : +- Scan parquet (505)
: : +- Sort (756)
: : +- Exchange (755)
: : +- SerializeFromObject (754)
: : +- MapElements (753)
: : +- DeserializeToObject (752)
: : +- ObjectHashAggregate (751)
: : +- ObjectHashAggregate (750)
: : +- Project (749)
: : +- BroadcastHashJoin LeftOuter BuildRight (748)
: : :- Project (745)
: : : +- SortMergeJoin LeftOuter (744)
: : : :- Sort (557)
: : : : +- Project (556)
: : : : +- Project (555)
: : : : +- Generate (554)
: : : : +- Generate (553)
: : : : +- Project (552)
: : : : +- Project (551)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (550)
: : : : :- Project (544)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (543)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (538)
: : : : : : :- Exchange (533)
: : : : : : : +- Scan parquet (181)
: : : : : : +- BroadcastExchange (537)
: : : : : : +- Project (536)
: : : : : : +- Exchange (535)
: : : : : : +- Project (534)
: : : : : : +- Scan parquet (185)
: : : : : +- BroadcastExchange (542)
: : : : : +- Exchange (541)
: : : : : +- Project (540)
: : : : : +- Scan parquet (539)
: : : : +- BroadcastExchange (549)
: : : : +- Exchange (548)
: : : : +- Project (547)
: : : : +- Filter (546)
: : : : +- Scan parquet (545)
: : : +- Sort (743)
: : : +- HashAggregate (742)
: : : +- Exchange (741)
: : : +- HashAggregate (740)
: : : +- Project (739)
: : : +- SortMergeJoin Inner (738)
: : : :- Sort (646)
: : : : +- Exchange (645)
: : : : +- Project (644)
: : : : +- SortMergeJoin Inner (643)
: : : : :- Sort (614)
: : : : : +- Exchange (613)
: : : : : +- Project (612)
: : : : : +- Filter (611)
: : : : : +- Generate (610)
: : : : : +- Filter (609)
: : : : : +- ObjectHashAggregate (608)
: : : : : +- ObjectHashAggregate (607)
: : : : : +- Generate (606)
: : : : : +- BroadcastHashJoin Inner BuildLeft (605)
: : : : : :- BroadcastExchange (562)
: : : : : : +- Exchange (561)
: : : : : : +- Project (560)
: : : : : : +- Filter (559)
: : : : : : +- Scan parquet (558)
: : : : : +- Project (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (572)
: : : : : : +- Project (571)
: : : : : : +- Generate (570)
: : : : : : +- Project (569)
: : : : : : +- Filter (568)
: : : : : : +- Generate (567)
: : : : : : +- Exchange (566)
: : : : : : +- Project (565)
: : : : : : +- Filter (564)
: : : : : : +- Scan parquet (563)
: : : : : +- HashAggregate (602)
: : : : : +- Exchange (601)
: : : : : +- HashAggregate (600)
: : : : : +- Project (599)
: : : : : +- Generate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (594)
: : : : : :- Project (588)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (587)
: : : : : : :- Project (582)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (581)
: : : : : : : :- Exchange (575)
: : : : : : : : +- Filter (574)
: : : : : : : : +- Scan parquet (573)
: : : : : : : +- BroadcastExchange (580)
: : : : : : : +- Project (579)
: : : : : : : +- Exchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Scan parquet (576)
: : : : : : +- BroadcastExchange (586)
: : : : : : +- Exchange (585)
: : : : : : +- Project (584)
: : : : : : +- Scan parquet (583)
: : : : : +- BroadcastExchange (593)
: : : : : +- Exchange (592)
: : : : : +- Project (591)
: : : : : +- Filter (590)
: : : : : +- Scan parquet (589)
: : : : +- Sort (642)
: : : : +- Project (641)
: : : : +- Generate (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (636)
: : : : :- Project (630)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (629)
: : : : : :- Project (624)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (623)
: : : : : : :- Exchange (617)
: : : : : : : +- Filter (616)
: : : : : : : +- Scan parquet (615)
: : : : : : +- BroadcastExchange (622)
: : : : : : +- Project (621)
: : : : : : +- Exchange (620)
: : : : : : +- Project (619)
: : : : : : +- Scan parquet (618)
: : : : : +- BroadcastExchange (628)
: : : : : +- Exchange (627)
: : : : : +- Project (626)
: : : : : +- Scan parquet (625)
: : : : +- BroadcastExchange (635)
: : : : +- Exchange (634)
: : : : +- Project (633)
: : : : +- Filter (632)
: : : : +- Scan parquet (631)
: : : +- Sort (737)
: : : +- Exchange (736)
: : : +- HashAggregate (735)
: : : +- HashAggregate (734)
: : : +- Project (733)
: : : +- SortMergeJoin Inner (732)
: : : :- Sort (703)
: : : : +- Exchange (702)
: : : : +- Project (701)
: : : : +- Filter (700)
: : : : +- Generate (699)
: : : : +- Filter (698)
: : : : +- ObjectHashAggregate (697)
: : : : +- ObjectHashAggregate (696)
: : : : +- Generate (695)
: : : : +- BroadcastHashJoin Inner BuildLeft (694)
: : : : :- BroadcastExchange (651)
: : : : : +- Exchange (650)
: : : : : +- Project (649)
: : : : : +- Filter (648)
: : : : : +- Scan parquet (647)
: : : : +- Project (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (661)
: : : : : +- Project (660)
: : : : : +- Generate (659)
: : : : : +- Project (658)
: : : : : +- Filter (657)
: : : : : +- Generate (656)
: : : : : +- Exchange (655)
: : : : : +- Project (654)
: : : : : +- Filter (653)
: : : : : +- Scan parquet (652)
: : : : +- HashAggregate (691)
: : : : +- Exchange (690)
: : : : +- HashAggregate (689)
: : : : +- Project (688)
: : : : +- Generate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (683)
: : : : :- Project (677)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (676)
: : : : : :- Project (671)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (670)
: : : : : : :- Exchange (664)
: : : : : : : +- Filter (663)
: : : : : : : +- Scan parquet (662)
: : : : : : +- BroadcastExchange (669)
: : : : : : +- Project (668)
: : : : : : +- Exchange (667)
: : : : : : +- Project (666)
: : : : : : +- Scan parquet (665)
: : : : : +- BroadcastExchange (675)
: : : : : +- Exchange (674)
: : : : : +- Project (673)
: : : : : +- Scan parquet (672)
: : : : +- BroadcastExchange (682)
: : : : +- Exchange (681)
: : : : +- Project (680)
: : : : +- Filter (679)
: : : : +- Scan parquet (678)
: : : +- Sort (731)
: : : +- Project (730)
: : : +- Generate (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- BroadcastHashJoin LeftOuter BuildRight (725)
: : : :- Project (719)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (718)
: : : : :- Project (713)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (712)
: : : : : :- Exchange (706)
: : : : : : +- Filter (705)
: : : : : : +- Scan parquet (704)
: : : : : +- BroadcastExchange (711)
: : : : : +- Project (710)
: : : : : +- Exchange (709)
: : : : : +- Project (708)
: : : : : +- Scan parquet (707)
: : : : +- BroadcastExchange (717)
: : : : +- Exchange (716)
: : : : +- Project (715)
: : : : +- Scan parquet (714)
: : : +- BroadcastExchange (724)
: : : +- Exchange (723)
: : : +- Project (722)
: : : +- Filter (721)
: : : +- Scan parquet (720)
: : +- BroadcastExchange (747)
: : +- Filter (746)
: : +- Scan parquet (269)
: +- Sort (804)
: +- Project (803)
: +- BroadcastHashJoin Inner BuildLeft (802)
: :- BroadcastExchange (771)
: : +- Project (770)
: : +- Filter (769)
: : +- Generate (768)
: : +- Project (767)
: : +- Filter (766)
: : +- Generate (765)
: : +- Exchange (764)
: : +- Project (763)
: : +- Filter (762)
: : +- Scan parquet (761)
: +- HashAggregate (801)
: +- Exchange (800)
: +- HashAggregate (799)
: +- Project (798)
: +- Generate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- BroadcastHashJoin LeftOuter BuildRight (793)
: :- Project (787)
: : +- BroadcastHashJoin LeftOuter BuildRight (786)
: : :- Project (781)
: : : +- BroadcastHashJoin LeftOuter BuildRight (780)
: : : :- Exchange (774)
: : : : +- Filter (773)
: : : : +- Scan parquet (772)
: : : +- BroadcastExchange (779)
: : : +- Project (778)
: : : +- Exchange (777)
: : : +- Project (776)
: : : +- Scan parquet (775)
: : +- BroadcastExchange (785)
: : +- Exchange (784)
: : +- Project (783)
: : +- Scan parquet (782)
: +- BroadcastExchange (792)
: +- Exchange (791)
: +- Project (790)
: +- Filter (789)
: +- Scan parquet (788)
+- BroadcastExchange (813)
+- Project (812)
+- Filter (811)
+- Generate (810)
+- Exchange (809)
+- Project (808)
+- Filter (807)
+- Scan parquet (310)
+- == Initial Plan ==
Execute InsertIntoHadoopFsRelationCommand (836)
+- WriteFiles (835)
+- Project (834)
+- Filter (833)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- AdaptiveSparkPlan (826)
+- == Final Plan ==
ResultQueryStage (4)
+- EmptyRelation (3)
+- Filter (unknown)
+- EmptyRelation (unknown)
+- == Initial Plan ==
Filter (825)
+- ObjectHashAggregate (824)
+- Exchange (823)
+- ObjectHashAggregate (822)
+- Project (821)
+- Window (820)
+- Sort (819)
+- Exchange (818)
+- Project (817)
+- InMemoryTableScan (5)
+- InMemoryRelation (6)
+- AdaptiveSparkPlan (816)
+- == Final Plan ==
ResultQueryStage (323)
+- Project (322)
+- * BroadcastHashJoin Inner BuildRight (321)
:- * Project (309)
: +- * BroadcastHashJoin Inner BuildLeft (308)
: :- BroadcastQueryStage (292), Statistics(sizeInBytes=2.0 MiB, rowCount=205)
: : +- BroadcastExchange (291)
: : +- AQEShuffleRead (290)
: : +- ShuffleQueryStage (289), Statistics(sizeInBytes=24.0 KiB, rowCount=205)
: : +- Exchange (288)
: : +- * Project (287)
: : +- * BroadcastHashJoin LeftOuter BuildRight (286)
: : :- AQEShuffleRead (180)
: : : +- ShuffleQueryStage (179), Statistics(sizeInBytes=24.0 KiB, rowCount=205)
: : : +- Exchange (178)
: : : +- * Project (177)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (176)
: : : :- * Project (44)
: : : : +- * Generate (43)
: : : : +- * Project (42)
: : : : +- * Generate (41)
: : : : +- Project (40)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (39)
: : : : :- * Project (30)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (29)
: : : : : :- * Project (21)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (20)
: : : : : : :- ShuffleQueryStage (11), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- Exchange (10)
: : : : : : : +- * Filter (9)
: : : : : : : +- * ColumnarToRow (8)
: : : : : : : +- Scan parquet (7)
: : : : : : +- BroadcastQueryStage (19), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (18)
: : : : : : +- * Project (17)
: : : : : : +- ShuffleQueryStage (16), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (15)
: : : : : : +- * Project (14)
: : : : : : +- * ColumnarToRow (13)
: : : : : : +- Scan parquet (12)
: : : : : +- BroadcastQueryStage (28), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (27)
: : : : : +- ShuffleQueryStage (26), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (25)
: : : : : +- * Project (24)
: : : : : +- * ColumnarToRow (23)
: : : : : +- Scan parquet (22)
: : : : +- BroadcastQueryStage (38), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (37)
: : : : +- ShuffleQueryStage (36), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (35)
: : : : +- * Project (34)
: : : : +- * Filter (33)
: : : : +- * ColumnarToRow (32)
: : : : +- Scan parquet (31)
: : : +- BroadcastQueryStage (175), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : +- BroadcastExchange (174)
: : : +- * HashAggregate (173)
: : : +- AQEShuffleRead (172)
: : : +- ShuffleQueryStage (171), Statistics(sizeInBytes=4.1 KiB, rowCount=44)
: : : +- Exchange (170)
: : : +- * HashAggregate (169)
: : : +- * Project (168)
: : : +- * BroadcastHashJoin Inner BuildLeft (167)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : : +- BroadcastExchange (138)
: : : : +- AQEShuffleRead (137)
: : : : +- ShuffleQueryStage (136), Statistics(sizeInBytes=4.8 KiB, rowCount=44)
: : : : +- Exchange (135)
: : : : +- * Project (134)
: : : : +- * BroadcastHashJoin Inner BuildLeft (133)
: : : : :- BroadcastQueryStage (111), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (110)
: : : : : +- AQEShuffleRead (109)
: : : : : +- ShuffleQueryStage (108), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (107)
: : : : : +- * Project (106)
: : : : : +- * Filter (105)
: : : : : +- * Generate (104)
: : : : : +- * Filter (103)
: : : : : +- ObjectHashAggregate (102)
: : : : : +- ObjectHashAggregate (101)
: : : : : +- Generate (100)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (99)
: : : : : :- BroadcastQueryStage (52), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (51)
: : : : : : +- ShuffleQueryStage (50), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (49)
: : : : : : +- * Project (48)
: : : : : : +- * Filter (47)
: : : : : : +- * ColumnarToRow (46)
: : : : : : +- Scan parquet (45)
: : : : : +- * Project (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (65), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (64)
: : : : : : +- * Project (63)
: : : : : : +- * Generate (62)
: : : : : : +- * Project (61)
: : : : : : +- * Filter (60)
: : : : : : +- * Generate (59)
: : : : : : +- ShuffleQueryStage (58), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (57)
: : : : : : +- * Project (56)
: : : : : : +- * Filter (55)
: : : : : : +- * ColumnarToRow (54)
: : : : : : +- Scan parquet (53)
: : : : : +- * HashAggregate (96)
: : : : : +- AQEShuffleRead (95)
: : : : : +- ShuffleQueryStage (94), Statistics(sizeInBytes=7.2 KiB, rowCount=153)
: : : : : +- Exchange (93)
: : : : : +- * HashAggregate (92)
: : : : : +- * Project (91)
: : : : : +- * Generate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- Project (87)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (86)
: : : : : :- * Project (83)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (82)
: : : : : : :- * Project (74)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (73)
: : : : : : : :- ShuffleQueryStage (70), Statistics(sizeInBytes=17.6 KiB, rowCount=205)
: : : : : : : : +- Exchange (69)
: : : : : : : : +- * Filter (68)
: : : : : : : : +- * ColumnarToRow (67)
: : : : : : : : +- Scan parquet (66)
: : : : : : : +- BroadcastQueryStage (72), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (71)
: : : : : : +- BroadcastQueryStage (81), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (80)
: : : : : : +- ShuffleQueryStage (79), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (78)
: : : : : : +- * Project (77)
: : : : : : +- * ColumnarToRow (76)
: : : : : : +- Scan parquet (75)
: : : : : +- BroadcastQueryStage (85), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (84)
: : : : +- * Project (132)
: : : : +- * Generate (131)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- Project (128)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (127)
: : : : :- * Project (124)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (123)
: : : : : :- * Project (120)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (119)
: : : : : : :- ShuffleQueryStage (116), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- Exchange (115)
: : : : : : : +- * Filter (114)
: : : : : : : +- * ColumnarToRow (113)
: : : : : : : +- Scan parquet (112)
: : : : : : +- BroadcastQueryStage (118), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (117)
: : : : : +- BroadcastQueryStage (122), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (121)
: : : : +- BroadcastQueryStage (126), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (125)
: : : +- AQEShuffleRead (166)
: : : +- ShuffleQueryStage (165), Statistics(sizeInBytes=4.8 KiB, rowCount=44)
: : : +- Exchange (164)
: : : +- * HashAggregate (163)
: : : +- * HashAggregate (162)
: : : +- * Project (161)
: : : +- * BroadcastHashJoin Inner BuildLeft (160)
: : : :- BroadcastQueryStage (141), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (140)
: : : +- * Project (159)
: : : +- * Generate (158)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- Project (155)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (154)
: : : :- * Project (151)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (150)
: : : : :- * Project (147)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (146)
: : : : : :- ShuffleQueryStage (143), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : +- ReusedExchange (142)
: : : : : +- BroadcastQueryStage (145), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (144)
: : : : +- BroadcastQueryStage (149), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (148)
: : : +- BroadcastQueryStage (153), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (152)
: : +- BroadcastQueryStage (285), Statistics(sizeInBytes=2.0 MiB, rowCount=205)
: : +- BroadcastExchange (284)
: : +- AQEShuffleRead (283)
: : +- ShuffleQueryStage (282), Statistics(sizeInBytes=17.6 KiB, rowCount=205)
: : +- Exchange (281)
: : +- * SerializeFromObject (280)
: : +- * MapElements (279)
: : +- * DeserializeToObject (278)
: : +- ObjectHashAggregate (277)
: : +- ObjectHashAggregate (276)
: : +- * Project (275)
: : +- * BroadcastHashJoin LeftOuter BuildRight (274)
: : :- * Project (268)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (267)
: : : :- * Project (206)
: : : : +- * Project (205)
: : : : +- * Generate (204)
: : : : +- * Generate (203)
: : : : +- * Project (202)
: : : : +- Project (201)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (200)
: : : : :- * Project (197)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (196)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (193)
: : : : : : :- ShuffleQueryStage (184), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- Exchange (183)
: : : : : : : +- * ColumnarToRow (182)
: : : : : : : +- Scan parquet (181)
: : : : : : +- BroadcastQueryStage (192), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (191)
: : : : : : +- * Project (190)
: : : : : : +- ShuffleQueryStage (189), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (188)
: : : : : : +- * Project (187)
: : : : : : +- * ColumnarToRow (186)
: : : : : : +- Scan parquet (185)
: : : : : +- BroadcastQueryStage (195), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (194)
: : : : +- BroadcastQueryStage (199), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (198)
: : : +- BroadcastQueryStage (266), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : +- BroadcastExchange (265)
: : : +- * HashAggregate (264)
: : : +- AQEShuffleRead (263)
: : : +- ShuffleQueryStage (262), Statistics(sizeInBytes=4.5 KiB, rowCount=44)
: : : +- Exchange (261)
: : : +- * HashAggregate (260)
: : : +- * Project (259)
: : : +- * BroadcastHashJoin Inner BuildLeft (258)
: : : :- BroadcastQueryStage (254), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
: : : : +- BroadcastExchange (253)
: : : : +- AQEShuffleRead (252)
: : : : +- ShuffleQueryStage (251), Statistics(sizeInBytes=5.2 KiB, rowCount=44)
: : : : +- Exchange (250)
: : : : +- * Project (249)
: : : : +- * BroadcastHashJoin Inner BuildLeft (248)
: : : : :- BroadcastQueryStage (229), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (228)
: : : : : +- AQEShuffleRead (227)
: : : : : +- ShuffleQueryStage (226), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (225)
: : : : : +- * Project (224)
: : : : : +- * Filter (223)
: : : : : +- * Generate (222)
: : : : : +- * Filter (221)
: : : : : +- ObjectHashAggregate (220)
: : : : : +- ObjectHashAggregate (219)
: : : : : +- Generate (218)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (217)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (207)
: : : : : +- * Project (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (210), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (209)
: : : : : +- * HashAggregate (214)
: : : : : +- AQEShuffleRead (213)
: : : : : +- ShuffleQueryStage (212), Statistics(sizeInBytes=7.2 KiB, rowCount=153)
: : : : : +- ReusedExchange (211)
: : : : +- * Project (247)
: : : : +- * Generate (246)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- Project (243)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (242)
: : : : :- * Project (239)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (238)
: : : : : :- * Project (235)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (234)
: : : : : : :- ShuffleQueryStage (231), Statistics(sizeInBytes=19.2 KiB, rowCount=205)
: : : : : : : +- ReusedExchange (230)
: : : : : : +- BroadcastQueryStage (233), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (232)
: : : : : +- BroadcastQueryStage (237), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (236)
: : : : +- BroadcastQueryStage (241), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (240)
: : : +- AQEShuffleRead (257)
: : : +- ShuffleQueryStage (256), Statistics(sizeInBytes=4.8 KiB, rowCount=44)
: : : +- ReusedExchange (255)
: : +- BroadcastQueryStage (273), Statistics(sizeInBytes=2.0 MiB, rowCount=205)
: : +- BroadcastExchange (272)
: : +- * Filter (271)
: : +- * ColumnarToRow (270)
: : +- Scan parquet (269)
: +- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (302), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (301)
: : +- * Project (300)
: : +- * Filter (299)
: : +- * Generate (298)
: : +- * Project (297)
: : +- * Filter (296)
: : +- * Generate (295)
: : +- ShuffleQueryStage (294), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (293)
: +- * HashAggregate (305)
: +- ShuffleQueryStage (304), Statistics(sizeInBytes=7.2 KiB, rowCount=153)
: +- ReusedExchange (303)
+- BroadcastQueryStage (320), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (319)
+- * Project (318)
+- * Filter (317)
+- * Generate (316)
+- ShuffleQueryStage (315), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (314)
+- * Project (313)
+- * Filter (312)
+- * ColumnarToRow (311)
+- Scan parquet (310)
+- == Initial Plan ==
Project (815)
+- BroadcastHashJoin Inner BuildRight (814)
:- Project (806)
: +- SortMergeJoin Inner (805)
: :- Sort (760)
: : +- Exchange (759)
: : +- Project (758)
: : +- SortMergeJoin LeftOuter (757)
: : :- Sort (532)
: : : +- Exchange (531)
: : : +- Project (530)
: : : +- SortMergeJoin LeftOuter (529)
: : : :- Sort (347)
: : : : +- Project (346)
: : : : +- Generate (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (341)
: : : : :- Project (336)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (335)
: : : : : :- Project (331)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (330)
: : : : : : :- Exchange (325)
: : : : : : : +- Filter (324)
: : : : : : : +- Scan parquet (7)
: : : : : : +- BroadcastExchange (329)
: : : : : : +- Project (328)
: : : : : : +- Exchange (327)
: : : : : : +- Project (326)
: : : : : : +- Scan parquet (12)
: : : : : +- BroadcastExchange (334)
: : : : : +- Exchange (333)
: : : : : +- Project (332)
: : : : : +- Scan parquet (22)
: : : : +- BroadcastExchange (340)
: : : : +- Exchange (339)
: : : : +- Project (338)
: : : : +- Filter (337)
: : : : +- Scan parquet (31)
: : : +- Sort (528)
: : : +- HashAggregate (527)
: : : +- Exchange (526)
: : : +- HashAggregate (525)
: : : +- Project (524)
: : : +- SortMergeJoin Inner (523)
: : : :- Sort (431)
: : : : +- Exchange (430)
: : : : +- Project (429)
: : : : +- SortMergeJoin Inner (428)
: : : : :- Sort (400)
: : : : : +- Exchange (399)
: : : : : +- Project (398)
: : : : : +- Filter (397)
: : : : : +- Generate (396)
: : : : : +- Filter (395)
: : : : : +- ObjectHashAggregate (394)
: : : : : +- ObjectHashAggregate (393)
: : : : : +- Generate (392)
: : : : : +- BroadcastHashJoin Inner BuildLeft (391)
: : : : : :- BroadcastExchange (351)
: : : : : : +- Exchange (350)
: : : : : : +- Project (349)
: : : : : : +- Filter (348)
: : : : : : +- Scan parquet (45)
: : : : : +- Project (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (360)
: : : : : : +- Project (359)
: : : : : : +- Generate (358)
: : : : : : +- Project (357)
: : : : : : +- Filter (356)
: : : : : : +- Generate (355)
: : : : : : +- Exchange (354)
: : : : : : +- Project (353)
: : : : : : +- Filter (352)
: : : : : : +- Scan parquet (53)
: : : : : +- HashAggregate (388)
: : : : : +- Exchange (387)
: : : : : +- HashAggregate (386)
: : : : : +- Project (385)
: : : : : +- Generate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (380)
: : : : : :- Project (374)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (373)
: : : : : : :- Project (369)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (368)
: : : : : : : :- Exchange (362)
: : : : : : : : +- Filter (361)
: : : : : : : : +- Scan parquet (66)
: : : : : : : +- BroadcastExchange (367)
: : : : : : : +- Project (366)
: : : : : : : +- Exchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Scan parquet (363)
: : : : : : +- BroadcastExchange (372)
: : : : : : +- Exchange (371)
: : : : : : +- Project (370)
: : : : : : +- Scan parquet (75)
: : : : : +- BroadcastExchange (379)
: : : : : +- Exchange (378)
: : : : : +- Project (377)
: : : : : +- Filter (376)
: : : : : +- Scan parquet (375)
: : : : +- Sort (427)
: : : : +- Project (426)
: : : : +- Generate (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (421)
: : : : :- Project (415)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (414)
: : : : : :- Project (409)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (408)
: : : : : : :- Exchange (402)
: : : : : : : +- Filter (401)
: : : : : : : +- Scan parquet (112)
: : : : : : +- BroadcastExchange (407)
: : : : : : +- Project (406)
: : : : : : +- Exchange (405)
: : : : : : +- Project (404)
: : : : : : +- Scan parquet (403)
: : : : : +- BroadcastExchange (413)
: : : : : +- Exchange (412)
: : : : : +- Project (411)
: : : : : +- Scan parquet (410)
: : : : +- BroadcastExchange (420)
: : : : +- Exchange (419)
: : : : +- Project (418)
: : : : +- Filter (417)
: : : : +- Scan parquet (416)
: : : +- Sort (522)
: : : +- Exchange (521)
: : : +- HashAggregate (520)
: : : +- HashAggregate (519)
: : : +- Project (518)
: : : +- SortMergeJoin Inner (517)
: : : :- Sort (488)
: : : : +- Exchange (487)
: : : : +- Project (486)
: : : : +- Filter (485)
: : : : +- Generate (484)
: : : : +- Filter (483)
: : : : +- ObjectHashAggregate (482)
: : : : +- ObjectHashAggregate (481)
: : : : +- Generate (480)
: : : : +- BroadcastHashJoin Inner BuildLeft (479)
: : : : :- BroadcastExchange (436)
: : : : : +- Exchange (435)
: : : : : +- Project (434)
: : : : : +- Filter (433)
: : : : : +- Scan parquet (432)
: : : : +- Project (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (446)
: : : : : +- Project (445)
: : : : : +- Generate (444)
: : : : : +- Project (443)
: : : : : +- Filter (442)
: : : : : +- Generate (441)
: : : : : +- Exchange (440)
: : : : : +- Project (439)
: : : : : +- Filter (438)
: : : : : +- Scan parquet (437)
: : : : +- HashAggregate (476)
: : : : +- Exchange (475)
: : : : +- HashAggregate (474)
: : : : +- Project (473)
: : : : +- Generate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (468)
: : : : :- Project (462)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (461)
: : : : : :- Project (456)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (455)
: : : : : : :- Exchange (449)
: : : : : : : +- Filter (448)
: : : : : : : +- Scan parquet (447)
: : : : : : +- BroadcastExchange (454)
: : : : : : +- Project (453)
: : : : : : +- Exchange (452)
: : : : : : +- Project (451)
: : : : : : +- Scan parquet (450)
: : : : : +- BroadcastExchange (460)
: : : : : +- Exchange (459)
: : : : : +- Project (458)
: : : : : +- Scan parquet (457)
: : : : +- BroadcastExchange (467)
: : : : +- Exchange (466)
: : : : +- Project (465)
: : : : +- Filter (464)
: : : : +- Scan parquet (463)
: : : +- Sort (516)
: : : +- Project (515)
: : : +- Generate (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- BroadcastHashJoin LeftOuter BuildRight (510)
: : : :- Project (504)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (503)
: : : : :- Project (498)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (497)
: : : : : :- Exchange (491)
: : : : : : +- Filter (490)
: : : : : : +- Scan parquet (489)
: : : : : +- BroadcastExchange (496)
: : : : : +- Project (495)
: : : : : +- Exchange (494)
: : : : : +- Project (493)
: : : : : +- Scan parquet (492)
: : : : +- BroadcastExchange (502)
: : : : +- Exchange (501)
: : : : +- Project (500)
: : : : +- Scan parquet (499)
: : : +- BroadcastExchange (509)
: : : +- Exchange (508)
: : : +- Project (507)
: : : +- Filter (506)
: : : +- Scan parquet (505)
: : +- Sort (756)
: : +- Exchange (755)
: : +- SerializeFromObject (754)
: : +- MapElements (753)
: : +- DeserializeToObject (752)
: : +- ObjectHashAggregate (751)
: : +- ObjectHashAggregate (750)
: : +- Project (749)
: : +- BroadcastHashJoin LeftOuter BuildRight (748)
: : :- Project (745)
: : : +- SortMergeJoin LeftOuter (744)
: : : :- Sort (557)
: : : : +- Project (556)
: : : : +- Project (555)
: : : : +- Generate (554)
: : : : +- Generate (553)
: : : : +- Project (552)
: : : : +- Project (551)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (550)
: : : : :- Project (544)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (543)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (538)
: : : : : : :- Exchange (533)
: : : : : : : +- Scan parquet (181)
: : : : : : +- BroadcastExchange (537)
: : : : : : +- Project (536)
: : : : : : +- Exchange (535)
: : : : : : +- Project (534)
: : : : : : +- Scan parquet (185)
: : : : : +- BroadcastExchange (542)
: : : : : +- Exchange (541)
: : : : : +- Project (540)
: : : : : +- Scan parquet (539)
: : : : +- BroadcastExchange (549)
: : : : +- Exchange (548)
: : : : +- Project (547)
: : : : +- Filter (546)
: : : : +- Scan parquet (545)
: : : +- Sort (743)
: : : +- HashAggregate (742)
: : : +- Exchange (741)
: : : +- HashAggregate (740)
: : : +- Project (739)
: : : +- SortMergeJoin Inner (738)
: : : :- Sort (646)
: : : : +- Exchange (645)
: : : : +- Project (644)
: : : : +- SortMergeJoin Inner (643)
: : : : :- Sort (614)
: : : : : +- Exchange (613)
: : : : : +- Project (612)
: : : : : +- Filter (611)
: : : : : +- Generate (610)
: : : : : +- Filter (609)
: : : : : +- ObjectHashAggregate (608)
: : : : : +- ObjectHashAggregate (607)
: : : : : +- Generate (606)
: : : : : +- BroadcastHashJoin Inner BuildLeft (605)
: : : : : :- BroadcastExchange (562)
: : : : : : +- Exchange (561)
: : : : : : +- Project (560)
: : : : : : +- Filter (559)
: : : : : : +- Scan parquet (558)
: : : : : +- Project (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (572)
: : : : : : +- Project (571)
: : : : : : +- Generate (570)
: : : : : : +- Project (569)
: : : : : : +- Filter (568)
: : : : : : +- Generate (567)
: : : : : : +- Exchange (566)
: : : : : : +- Project (565)
: : : : : : +- Filter (564)
: : : : : : +- Scan parquet (563)
: : : : : +- HashAggregate (602)
: : : : : +- Exchange (601)
: : : : : +- HashAggregate (600)
: : : : : +- Project (599)
: : : : : +- Generate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (594)
: : : : : :- Project (588)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (587)
: : : : : : :- Project (582)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (581)
: : : : : : : :- Exchange (575)
: : : : : : : : +- Filter (574)
: : : : : : : : +- Scan parquet (573)
: : : : : : : +- BroadcastExchange (580)
: : : : : : : +- Project (579)
: : : : : : : +- Exchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Scan parquet (576)
: : : : : : +- BroadcastExchange (586)
: : : : : : +- Exchange (585)
: : : : : : +- Project (584)
: : : : : : +- Scan parquet (583)
: : : : : +- BroadcastExchange (593)
: : : : : +- Exchange (592)
: : : : : +- Project (591)
: : : : : +- Filter (590)
: : : : : +- Scan parquet (589)
: : : : +- Sort (642)
: : : : +- Project (641)
: : : : +- Generate (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (636)
: : : : :- Project (630)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (629)
: : : : : :- Project (624)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (623)
: : : : : : :- Exchange (617)
: : : : : : : +- Filter (616)
: : : : : : : +- Scan parquet (615)
: : : : : : +- BroadcastExchange (622)
: : : : : : +- Project (621)
: : : : : : +- Exchange (620)
: : : : : : +- Project (619)
: : : : : : +- Scan parquet (618)
: : : : : +- BroadcastExchange (628)
: : : : : +- Exchange (627)
: : : : : +- Project (626)
: : : : : +- Scan parquet (625)
: : : : +- BroadcastExchange (635)
: : : : +- Exchange (634)
: : : : +- Project (633)
: : : : +- Filter (632)
: : : : +- Scan parquet (631)
: : : +- Sort (737)
: : : +- Exchange (736)
: : : +- HashAggregate (735)
: : : +- HashAggregate (734)
: : : +- Project (733)
: : : +- SortMergeJoin Inner (732)
: : : :- Sort (703)
: : : : +- Exchange (702)
: : : : +- Project (701)
: : : : +- Filter (700)
: : : : +- Generate (699)
: : : : +- Filter (698)
: : : : +- ObjectHashAggregate (697)
: : : : +- ObjectHashAggregate (696)
: : : : +- Generate (695)
: : : : +- BroadcastHashJoin Inner BuildLeft (694)
: : : : :- BroadcastExchange (651)
: : : : : +- Exchange (650)
: : : : : +- Project (649)
: : : : : +- Filter (648)
: : : : : +- Scan parquet (647)
: : : : +- Project (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (661)
: : : : : +- Project (660)
: : : : : +- Generate (659)
: : : : : +- Project (658)
: : : : : +- Filter (657)
: : : : : +- Generate (656)
: : : : : +- Exchange (655)
: : : : : +- Project (654)
: : : : : +- Filter (653)
: : : : : +- Scan parquet (652)
: : : : +- HashAggregate (691)
: : : : +- Exchange (690)
: : : : +- HashAggregate (689)
: : : : +- Project (688)
: : : : +- Generate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (683)
: : : : :- Project (677)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (676)
: : : : : :- Project (671)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (670)
: : : : : : :- Exchange (664)
: : : : : : : +- Filter (663)
: : : : : : : +- Scan parquet (662)
: : : : : : +- BroadcastExchange (669)
: : : : : : +- Project (668)
: : : : : : +- Exchange (667)
: : : : : : +- Project (666)
: : : : : : +- Scan parquet (665)
: : : : : +- BroadcastExchange (675)
: : : : : +- Exchange (674)
: : : : : +- Project (673)
: : : : : +- Scan parquet (672)
: : : : +- BroadcastExchange (682)
: : : : +- Exchange (681)
: : : : +- Project (680)
: : : : +- Filter (679)
: : : : +- Scan parquet (678)
: : : +- Sort (731)
: : : +- Project (730)
: : : +- Generate (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- BroadcastHashJoin LeftOuter BuildRight (725)
: : : :- Project (719)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (718)
: : : : :- Project (713)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (712)
: : : : : :- Exchange (706)
: : : : : : +- Filter (705)
: : : : : : +- Scan parquet (704)
: : : : : +- BroadcastExchange (711)
: : : : : +- Project (710)
: : : : : +- Exchange (709)
: : : : : +- Project (708)
: : : : : +- Scan parquet (707)
: : : : +- BroadcastExchange (717)
: : : : +- Exchange (716)
: : : : +- Project (715)
: : : : +- Scan parquet (714)
: : : +- BroadcastExchange (724)
: : : +- Exchange (723)
: : : +- Project (722)
: : : +- Filter (721)
: : : +- Scan parquet (720)
: : +- BroadcastExchange (747)
: : +- Filter (746)
: : +- Scan parquet (269)
: +- Sort (804)
: +- Project (803)
: +- BroadcastHashJoin Inner BuildLeft (802)
: :- BroadcastExchange (771)
: : +- Project (770)
: : +- Filter (769)
: : +- Generate (768)
: : +- Project (767)
: : +- Filter (766)
: : +- Generate (765)
: : +- Exchange (764)
: : +- Project (763)
: : +- Filter (762)
: : +- Scan parquet (761)
: +- HashAggregate (801)
: +- Exchange (800)
: +- HashAggregate (799)
: +- Project (798)
: +- Generate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- BroadcastHashJoin LeftOuter BuildRight (793)
: :- Project (787)
: : +- BroadcastHashJoin LeftOuter BuildRight (786)
: : :- Project (781)
: : : +- BroadcastHashJoin LeftOuter BuildRight (780)
: : : :- Exchange (774)
: : : : +- Filter (773)
: : : : +- Scan parquet (772)
: : : +- BroadcastExchange (779)
: : : +- Project (778)
: : : +- Exchange (777)
: : : +- Project (776)
: : : +- Scan parquet (775)
: : +- BroadcastExchange (785)
: : +- Exchange (784)
: : +- Project (783)
: : +- Scan parquet (782)
: +- BroadcastExchange (792)
: +- Exchange (791)
: +- Project (790)
: +- Filter (789)
: +- Scan parquet (788)
+- BroadcastExchange (813)
+- Project (812)
+- Filter (811)
+- Generate (810)
+- Exchange (809)
+- Project (808)
+- Filter (807)
+- Scan parquet (310)
(1) InMemoryTableScan
Output [3]: [openGoodsMovements#75051, plant#70763, product#70764]
Arguments: [openGoodsMovements#75051, plant#70763, product#70764], [(size(transform(openGoodsMovements#75051, lambdafunction(struct(documentType, PO, documentId, lambda x_28#90539.documentId, documentUuid, lambda x_28#90539.documentUuid, openQty, lambda x_28#90539.openQty, qtyUomUuid, lambda x_28#90539.qtyUomUuid, qtyUom, lambda x_28#90539.qtyUom, plannedDeliveryDatetime, lambda x_28#90539.availabilityDatetime), lambda x_28#90539, false)), false) > 0)]
(2) InMemoryRelation
Arguments: [product#70764, plant#70763, openGoodsMovements#75051], StorageLevel(disk, memory, 2 replicas)
(3) EmptyRelation
Output [3]: [product#70764, plant#70763, openGoodsMovements#75051]
Arguments: [plan_id=129538]
(4) ResultQueryStage
Output [3]: [product#70764, plant#70763, openGoodsMovements#75051]
Arguments: 2
(5) InMemoryTableScan
Output [10]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityDenominator#70772, quantityNumerator#70771, quantityUom#70768, quantityUomUuid#70769, scheduleLines#70774, validConfirmations#70964]
Arguments: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityDenominator#70772, quantityNumerator#70771, quantityUom#70768, quantityUomUuid#70769, scheduleLines#70774, validConfirmations#70964]
(6) InMemoryRelation
Arguments: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776, stockDatetime#69867, timezonecode#69080, validConfirmations#70964], StorageLevel(disk, memory, deserialized, 1 replicas)
(7) Scan parquet
Output [3]: [product#69015, plant#69016, orderDateTime#69017]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(8) ColumnarToRow [codegen id : 1]
Input [3]: [product#69015, plant#69016, orderDateTime#69017]
(9) Filter [codegen id : 1]
Input [3]: [product#69015, plant#69016, orderDateTime#69017]
Condition : (isnotnull(plant#69016) AND isnotnull(product#69015))
(10) Exchange
Input [3]: [product#69015, plant#69016, orderDateTime#69017]
Arguments: hashpartitioning(product#69015, plant#69016, 25), REPARTITION_BY_NUM, [plan_id=45442]
(11) ShuffleQueryStage
Output [3]: [product#69015, plant#69016, orderDateTime#69017]
Arguments: 0
(12) Scan parquet
Output [4]: [plant#69030, product#69032, productionAspect#69035, systemId#69026]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(13) ColumnarToRow [codegen id : 2]
Input [4]: [plant#69030, product#69032, productionAspect#69035, systemId#69026]
(14) Project [codegen id : 2]
Output [4]: [plant#69030.internalRefUUID AS _extract_internalRefUUID#71356, product#69032.internalRefUUID AS _extract_internalRefUUID#71355, productionAspect#69035.productMovementPlants.unitOfIssue.code AS _extract_code#71358, productionAspect#69035.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71359]
Input [4]: [plant#69030, product#69032, productionAspect#69035, systemId#69026]
(15) Exchange
Input [4]: [_extract_internalRefUUID#71356, _extract_internalRefUUID#71355, _extract_code#71358, _extract_internalRefUUID#71359]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=45462]
(16) ShuffleQueryStage
Output [4]: [_extract_internalRefUUID#71356, _extract_internalRefUUID#71355, _extract_code#71358, _extract_internalRefUUID#71359]
Arguments: 1
(17) Project [codegen id : 56]
Output [4]: [_extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
Input [4]: [_extract_internalRefUUID#71356, _extract_internalRefUUID#71355, _extract_code#71358, _extract_internalRefUUID#71359]
(18) BroadcastExchange
Input [4]: [_extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=60476]
(19) BroadcastQueryStage
Output [4]: [_extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
Arguments: 119
(20) BroadcastHashJoin [codegen id : 106]
Left keys [4]: [coalesce(product#69015, ), isnull(product#69015), coalesce(plant#69016, ), isnull(plant#69016)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71355, ), isnull(_extract_internalRefUUID#71355), coalesce(_extract_internalRefUUID#71356, ), isnull(_extract_internalRefUUID#71356)]
Join type: LeftOuter
Join condition: None
(21) Project [codegen id : 106]
Output [5]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359]
Input [7]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
(22) Scan parquet
Output [3]: [internalUUID#69050, address#69056, systemId#69049]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string,address:struct<timeZone:struct<code:string>>>
(23) ColumnarToRow [codegen id : 3]
Input [3]: [internalUUID#69050, address#69056, systemId#69049]
(24) Project [codegen id : 3]
Output [2]: [internalUUID#69050, address#69056.timeZone.code AS timezonecode#69073]
Input [3]: [internalUUID#69050, address#69056, systemId#69049]
(25) Exchange
Input [2]: [internalUUID#69050, timezonecode#69073]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=45486]
(26) ShuffleQueryStage
Output [2]: [internalUUID#69050, timezonecode#69073]
Arguments: 2
(27) BroadcastExchange
Input [2]: [internalUUID#69050, timezonecode#69073]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=56025]
(28) BroadcastQueryStage
Output [2]: [internalUUID#69050, timezonecode#69073]
Arguments: 108
(29) BroadcastHashJoin [codegen id : 106]
Left keys [2]: [coalesce(plant#69016, ), isnull(plant#69016)]
Right keys [2]: [coalesce(internalUUID#69050, ), isnull(internalUUID#69050)]
Join type: LeftOuter
Join condition: None
(30) Project [codegen id : 106]
Output [6]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, CASE WHEN isnotnull(timezonecode#69073) THEN timezonecode#69073 ELSE UTC END AS timezonecode#69080]
Input [7]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, internalUUID#69050, timezonecode#69073]
(31) Scan parquet
Output [4]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088, systemId#69081]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(32) ColumnarToRow [codegen id : 4]
Input [4]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088, systemId#69081]
(33) Filter [codegen id : 4]
Input [4]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088, systemId#69081]
Condition : isnotnull(internalUUID#69082)
(34) Project [codegen id : 4]
Output [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Input [4]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088, systemId#69081]
(35) Exchange
Input [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=45516]
(36) ShuffleQueryStage
Output [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Arguments: 3
(37) BroadcastExchange
Input [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=54084]
(38) BroadcastQueryStage
Output [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Arguments: 97
(39) BroadcastHashJoin [codegen id : 106]
Left keys [1]: [product#69015]
Right keys [1]: [internalUUID#69082]
Join type: LeftOuter
Join condition: None
(40) Project
Output [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filter(unitOfMeasureConversions#69088, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71358) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71359)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69086.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69086.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69108]
Input [9]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, timezonecode#69080, internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
(41) Generate [codegen id : 107]
Input [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108]
Arguments: explode(filteredUnitOfMeasureConversions#69108.quantityNumerator), [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108], true, [quantityNumerator#69123]
(42) Project [codegen id : 107]
Output [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108.quantityDenominator AS _extract_quantityDenominator#70967]
Input [6]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108, quantityNumerator#69123]
(43) Generate [codegen id : 107]
Input [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, _extract_quantityDenominator#70967]
Arguments: explode(_extract_quantityDenominator#70967), [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080], true, [quantityDenominator#69130]
(44) Project [codegen id : 107]
Output [4]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080]
Input [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, quantityDenominator#69130]
(45) Scan parquet
Output [6]: [internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, systemId#69251]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(46) ColumnarToRow [codegen id : 5]
Input [6]: [internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, systemId#69251]
(47) Filter [codegen id : 5]
Input [6]: [internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, systemId#69251]
Condition : (isnotnull(plant#69254.internalRefUUID) AND isnotnull(product#69253.internalRefUUID))
(48) Project [codegen id : 5]
Output [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Input [6]: [internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, systemId#69251]
(49) Exchange
Input [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=45551]
(50) ShuffleQueryStage
Output [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Arguments: 4
(51) BroadcastExchange
Input [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=80250]
(52) BroadcastQueryStage
Output [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Arguments: 148
(53) Scan parquet
Output [2]: [marketUnit#69147, distributionCenters#69161]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(54) ColumnarToRow [codegen id : 6]
Input [2]: [marketUnit#69147, distributionCenters#69161]
(55) Filter [codegen id : 6]
Input [2]: [marketUnit#69147, distributionCenters#69161]
Condition : (((marketUnit#69147 <=> Daily_grocery) AND (size(distributionCenters#69161, true) > 0)) AND isnotnull(distributionCenters#69161))
(56) Project [codegen id : 6]
Output [1]: [distributionCenters#69161]
Input [2]: [marketUnit#69147, distributionCenters#69161]
(57) Exchange
Input [1]: [distributionCenters#69161]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=45575]
(58) ShuffleQueryStage
Output [1]: [distributionCenters#69161]
Arguments: 5
(59) Generate [codegen id : 57]
Input [1]: [distributionCenters#69161]
Arguments: explode(distributionCenters#69161), false, [distributionCenters#69184]
(60) Filter [codegen id : 57]
Input [1]: [distributionCenters#69184]
Condition : (((size(distributionCenters#69184.storageLocations, true) > 0) AND isnotnull(distributionCenters#69184.storageLocations)) AND isnotnull(distributionCenters#69184.internalRefUUID))
(61) Project [codegen id : 57]
Output [2]: [distributionCenters#69184.internalRefUUID AS _extract_internalRefUUID#70975, distributionCenters#69184.storageLocations AS _extract_storageLocations#70976]
Input [1]: [distributionCenters#69184]
(62) Generate [codegen id : 57]
Input [2]: [_extract_internalRefUUID#70975, _extract_storageLocations#70976]
Arguments: explode(_extract_storageLocations#70976), [_extract_internalRefUUID#70975], false, [storageLocations#69198]
(63) Project [codegen id : 57]
Output [3]: [_extract_internalRefUUID#70975 AS plantUuid#69189, storageLocations#69198.id AS storageLocId#69201, storageLocations#69198.internalRefUUID AS storageLocUuid#69202]
Input [2]: [_extract_internalRefUUID#70975, storageLocations#69198]
(64) BroadcastExchange
Input [3]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=60501]
(65) BroadcastQueryStage
Output [3]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Arguments: 120
(66) Scan parquet
Output [2]: [product#69783, plant#69784]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(67) ColumnarToRow [codegen id : 7]
Input [2]: [product#69783, plant#69784]
(68) Filter [codegen id : 7]
Input [2]: [product#69783, plant#69784]
Condition : isnotnull(plant#69784)
(69) Exchange
Input [2]: [product#69783, plant#69784]
Arguments: hashpartitioning(product#69783, plant#69784, 25), REPARTITION_BY_NUM, [plan_id=45603]
(70) ShuffleQueryStage
Output [2]: [product#69783, plant#69784]
Arguments: 6
(71) ReusedExchange [Reuses operator id: 18]
Output [4]: [_extract_code#71364, _extract_internalRefUUID#71365, _extract_internalRefUUID#71363, _extract_internalRefUUID#71366]
(72) BroadcastQueryStage
Output [4]: [_extract_code#71364, _extract_internalRefUUID#71365, _extract_internalRefUUID#71363, _extract_internalRefUUID#71366]
Arguments: 122
(73) BroadcastHashJoin [codegen id : 73]
Left keys [4]: [coalesce(product#69783, ), isnull(product#69783), coalesce(plant#69784, ), isnull(plant#69784)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71363, ), isnull(_extract_internalRefUUID#71363), coalesce(_extract_internalRefUUID#71366, ), isnull(_extract_internalRefUUID#71366)]
Join type: LeftOuter
Join condition: None
(74) Project [codegen id : 73]
Output [4]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365]
Input [6]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365, _extract_internalRefUUID#71363, _extract_internalRefUUID#71366]
(75) Scan parquet
Output [2]: [internalUUID#69810, systemId#69809]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(76) ColumnarToRow [codegen id : 9]
Input [2]: [internalUUID#69810, systemId#69809]
(77) Project [codegen id : 9]
Output [1]: [internalUUID#69810]
Input [2]: [internalUUID#69810, systemId#69809]
(78) Exchange
Input [1]: [internalUUID#69810]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=45644]
(79) ShuffleQueryStage
Output [1]: [internalUUID#69810]
Arguments: 9
(80) BroadcastExchange
Input [1]: [internalUUID#69810]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=57778]
(81) BroadcastQueryStage
Output [1]: [internalUUID#69810]
Arguments: 110
(82) BroadcastHashJoin [codegen id : 73]
Left keys [2]: [coalesce(plant#69784, ), isnull(plant#69784)]
Right keys [2]: [coalesce(internalUUID#69810, ), isnull(internalUUID#69810)]
Join type: LeftOuter
Join condition: None
(83) Project [codegen id : 73]
Output [4]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365]
Input [5]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365, internalUUID#69810]
(84) ReusedExchange [Reuses operator id: 37]
Output [3]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
(85) BroadcastQueryStage
Output [3]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
Arguments: 98
(86) BroadcastHashJoin [codegen id : 73]
Left keys [1]: [product#69783]
Right keys [1]: [internalUUID#69838]
Join type: LeftOuter
Join condition: None
(87) Project
Output [2]: [plant#69784, filter(unitOfMeasureConversions#69844, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71364) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71365)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69842.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69842.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69850]
Input [7]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365, internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
(88) Generate [codegen id : 74]
Input [2]: [plant#69784, filteredUnitOfMeasureConversions#69850]
Arguments: explode(filteredUnitOfMeasureConversions#69850.quantityNumerator), [plant#69784, filteredUnitOfMeasureConversions#69850], true, [quantityNumerator#69852]
(89) Project [codegen id : 74]
Output [2]: [plant#69784, filteredUnitOfMeasureConversions#69850.quantityDenominator AS _extract_quantityDenominator#70979]
Input [3]: [plant#69784, filteredUnitOfMeasureConversions#69850, quantityNumerator#69852]
(90) Generate [codegen id : 74]
Input [2]: [plant#69784, _extract_quantityDenominator#70979]
Arguments: explode(_extract_quantityDenominator#70979), [plant#69784], true, [quantityDenominator#69853]
(91) Project [codegen id : 74]
Output [1]: [plant#69784]
Input [2]: [plant#69784, quantityDenominator#69853]
(92) HashAggregate [codegen id : 74]
Input [1]: [plant#69784]
Keys [1]: [plant#69784]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69784]
(93) Exchange
Input [1]: [plant#69784]
Arguments: hashpartitioning(plant#69784, 25), ENSURE_REQUIREMENTS, [plan_id=91603]
(94) ShuffleQueryStage
Output [1]: [plant#69784]
Arguments: 152
(95) AQEShuffleRead
Input [1]: [plant#69784]
Arguments: coalesced
(96) HashAggregate
Input [1]: [plant#69784]
Keys [1]: [plant#69784]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69784]
(97) BroadcastHashJoin
Left keys [1]: [plantUuid#69189]
Right keys [1]: [plant#69784]
Join type: Inner
Join condition: None
(98) Project
Output [3]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Input [4]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202, plant#69784]
(99) BroadcastHashJoin [codegen id : 83]
Left keys [2]: [coalesce(plant#69254.internalRefUUID, ), isnull(plant#69254.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#69189, ), isnull(plantUuid#69189)]
Join type: Inner
Join condition: None
(100) Generate
Input [9]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Arguments: explode(filter(stockQuantities#69256, lambdafunction(((lambda x#69270.storageLocation.internalRefUuid = storageLocUuid#69202) AND (lambda x#69270.storageLocation.Id = storageLocId#69201)), lambda x#69270, false))), [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, plantUuid#69189], false, [stockQuantity#69271]
(101) ObjectHashAggregate
Input [7]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, plantUuid#69189, stockQuantity#69271]
Keys [5]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251]
Functions [2]: [partial_first(calculationDateTime#69255, false), partial_collect_list(stockQuantity#69271, 0, 0)]
Aggregate Attributes [3]: [first#73908, valueSet#73909, buf#73910]
Results [8]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251, first#73911, valueSet#73912, buf#73913]
(102) ObjectHashAggregate
Input [8]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251, first#73911, valueSet#73912, buf#73913]
Keys [5]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251]
Functions [2]: [first(calculationDateTime#69255, false), collect_list(stockQuantity#69271, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#69255)()#69321, collect_list(stockQuantity#69271, 0, 0)#69322]
Results [4]: [product#69253, plant#69254, first(calculationDateTime#69255)()#69321 AS calculationDateTime#69308, collect_list(stockQuantity#69271, 0, 0)#69322 AS stockQuantities#69309]
(103) Filter [codegen id : 84]
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantities#69309]
Condition : ((size(stockQuantities#69309, true) > 0) AND isnotnull(calculationDateTime#69308))
(104) Generate [codegen id : 84]
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantities#69309]
Arguments: explode(stockQuantities#69309), [product#69253, plant#69254, calculationDateTime#69308], false, [stockQuantity#69367]
(105) Filter [codegen id : 84]
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantity#69367]
Condition : (((isnotnull(stockQuantity#69367.specialStockIndicator.code) AND isnotnull(stockQuantity#69367.stockType.code)) AND (stockQuantity#69367.specialStockIndicator.code = )) AND (stockQuantity#69367.stockType.code = 01))
(106) Project [codegen id : 84]
Output [4]: [plant#69254.internalRefUUID AS plantUuid#69370, product#69253.internalRefUUID AS productUuid#69372, calculationDateTime#69308, stockQuantity#69367.storagelocation.id AS storageLocId#69373]
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantity#69367]
(107) Exchange
Input [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: hashpartitioning(productUuid#69372, plantUuid#69370, 25), ENSURE_REQUIREMENTS, [plan_id=114438]
(108) ShuffleQueryStage
Output [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: 161
(109) AQEShuffleRead
Input [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: local
(110) BroadcastExchange
Input [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true], input[1, string, true]),false), [plan_id=121339]
(111) BroadcastQueryStage
Output [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: 168
(112) Scan parquet
Output [3]: [product#69391, plant#69392, orderDateTime#69393]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(113) ColumnarToRow [codegen id : 11]
Input [3]: [product#69391, plant#69392, orderDateTime#69393]
(114) Filter [codegen id : 11]
Input [3]: [product#69391, plant#69392, orderDateTime#69393]
Condition : ((isnotnull(orderDateTime#69393) AND isnotnull(plant#69392)) AND isnotnull(product#69391))
(115) Exchange
Input [3]: [product#69391, plant#69392, orderDateTime#69393]
Arguments: hashpartitioning(product#69391, plant#69392, 25), REPARTITION_BY_NUM, [plan_id=45719]
(116) ShuffleQueryStage
Output [3]: [product#69391, plant#69392, orderDateTime#69393]
Arguments: 12
(117) ReusedExchange [Reuses operator id: 18]
Output [4]: [_extract_code#71372, _extract_internalRefUUID#71373, _extract_internalRefUUID#71371, _extract_internalRefUUID#71374]
(118) BroadcastQueryStage
Output [4]: [_extract_code#71372, _extract_internalRefUUID#71373, _extract_internalRefUUID#71371, _extract_internalRefUUID#71374]
Arguments: 124
(119) BroadcastHashJoin [codegen id : 93]
Left keys [4]: [coalesce(product#69391, ), isnull(product#69391), coalesce(plant#69392, ), isnull(plant#69392)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71371, ), isnull(_extract_internalRefUUID#71371), coalesce(_extract_internalRefUUID#71374, ), isnull(_extract_internalRefUUID#71374)]
Join type: LeftOuter
Join condition: None
(120) Project [codegen id : 93]
Output [5]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373]
Input [7]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373, _extract_internalRefUUID#71371, _extract_internalRefUUID#71374]
(121) ReusedExchange [Reuses operator id: 80]
Output [1]: [internalUUID#69405]
(122) BroadcastQueryStage
Output [1]: [internalUUID#69405]
Arguments: 111
(123) BroadcastHashJoin [codegen id : 93]
Left keys [2]: [coalesce(plant#69392, ), isnull(plant#69392)]
Right keys [2]: [coalesce(internalUUID#69405, ), isnull(internalUUID#69405)]
Join type: LeftOuter
Join condition: None
(124) Project [codegen id : 93]
Output [5]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373]
Input [6]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373, internalUUID#69405]
(125) ReusedExchange [Reuses operator id: 37]
Output [3]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
(126) BroadcastQueryStage
Output [3]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
Arguments: 99
(127) BroadcastHashJoin [codegen id : 93]
Left keys [1]: [product#69391]
Right keys [1]: [internalUUID#69419]
Join type: LeftOuter
Join condition: None
(128) Project
Output [4]: [product#69391, plant#69392, orderDateTime#69393, filter(unitOfMeasureConversions#69425, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71372) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71373)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69423.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69423.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69430]
Input [8]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373, internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
(129) Generate
Input [4]: [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430]
Arguments: explode(filteredUnitOfMeasureConversions#69430.quantityNumerator), [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430], true, [quantityNumerator#69431]
(130) Project
Output [4]: [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430.quantityDenominator AS _extract_quantityDenominator#70987]
Input [5]: [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430, quantityNumerator#69431]
(131) Generate
Input [4]: [product#69391, plant#69392, orderDateTime#69393, _extract_quantityDenominator#70987]
Arguments: explode(_extract_quantityDenominator#70987), [product#69391, plant#69392, orderDateTime#69393], true, [quantityDenominator#69432]
(132) Project
Output [3]: [product#69391, plant#69392, orderDateTime#69393]
Input [4]: [product#69391, plant#69392, orderDateTime#69393, quantityDenominator#69432]
(133) BroadcastHashJoin [codegen id : 94]
Left keys [2]: [plantUuid#69370, productUuid#69372]
Right keys [2]: [plant#69392, product#69391]
Join type: Inner
Join condition: (calculationDateTime#69308 <= orderDateTime#69393)
(134) Project [codegen id : 94]
Output [4]: [plantUuid#69370 AS plant#69448, productUuid#69372 AS product#69449, storageLocId#69373, calculationDateTime#69308]
Input [7]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373, product#69391, plant#69392, orderDateTime#69393]
(135) Exchange
Input [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: hashpartitioning(coalesce(plant#69448, ), isnull(plant#69448), coalesce(product#69449, ), isnull(product#69449), coalesce(storageLocId#69373, ), isnull(storageLocId#69373), coalesce(calculationDateTime#69308, 1970-01-01 00:00:00), isnull(calculationDateTime#69308), 25), ENSURE_REQUIREMENTS, [plan_id=122715]
(136) ShuffleQueryStage
Output [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: 174
(137) AQEShuffleRead
Input [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: local
(138) BroadcastExchange
Input [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true]), coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, timestamp, true], 1970-01-01 00:00:00), isnull(input[3, timestamp, true])),false), [plan_id=125325]
(139) BroadcastQueryStage
Output [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: 179
(140) ReusedExchange [Reuses operator id: 110]
Output [4]: [plantUuid#69677, productUuid#69679, calculationDateTime#69673, storageLocId#69680]
(141) BroadcastQueryStage
Output [4]: [plantUuid#69677, productUuid#69679, calculationDateTime#69673, storageLocId#69680]
Arguments: 170
(142) ReusedExchange [Reuses operator id: 115]
Output [3]: [product#69685, plant#69686, orderDateTime#69687]
(143) ShuffleQueryStage
Output [3]: [product#69685, plant#69686, orderDateTime#69687]
Arguments: 32
(144) ReusedExchange [Reuses operator id: 18]
Output [4]: [_extract_code#71385, _extract_internalRefUUID#71386, _extract_internalRefUUID#71384, _extract_internalRefUUID#71383]
(145) BroadcastQueryStage
Output [4]: [_extract_code#71385, _extract_internalRefUUID#71386, _extract_internalRefUUID#71384, _extract_internalRefUUID#71383]
Arguments: 130
(146) BroadcastHashJoin [codegen id : 95]
Left keys [4]: [coalesce(product#69685, ), isnull(product#69685), coalesce(plant#69686, ), isnull(plant#69686)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71384, ), isnull(_extract_internalRefUUID#71384), coalesce(_extract_internalRefUUID#71383, ), isnull(_extract_internalRefUUID#71383)]
Join type: LeftOuter
Join condition: None
(147) Project [codegen id : 95]
Output [5]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386]
Input [7]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386, _extract_internalRefUUID#71384, _extract_internalRefUUID#71383]
(148) ReusedExchange [Reuses operator id: 80]
Output [1]: [internalUUID#69699]
(149) BroadcastQueryStage
Output [1]: [internalUUID#69699]
Arguments: 113
(150) BroadcastHashJoin [codegen id : 95]
Left keys [2]: [coalesce(plant#69686, ), isnull(plant#69686)]
Right keys [2]: [coalesce(internalUUID#69699, ), isnull(internalUUID#69699)]
Join type: LeftOuter
Join condition: None
(151) Project [codegen id : 95]
Output [5]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386]
Input [6]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386, internalUUID#69699]
(152) ReusedExchange [Reuses operator id: 37]
Output [3]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
(153) BroadcastQueryStage
Output [3]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
Arguments: 101
(154) BroadcastHashJoin [codegen id : 95]
Left keys [1]: [product#69685]
Right keys [1]: [internalUUID#69713]
Join type: LeftOuter
Join condition: None
(155) Project
Output [4]: [product#69685, plant#69686, orderDateTime#69687, filter(unitOfMeasureConversions#69719, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71385) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71386)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69717.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69717.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69724]
Input [8]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386, internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
(156) Generate
Input [4]: [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724]
Arguments: explode(filteredUnitOfMeasureConversions#69724.quantityNumerator), [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724], true, [quantityNumerator#69725]
(157) Project
Output [4]: [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724.quantityDenominator AS _extract_quantityDenominator#71006]
Input [5]: [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724, quantityNumerator#69725]
(158) Generate
Input [4]: [product#69685, plant#69686, orderDateTime#69687, _extract_quantityDenominator#71006]
Arguments: explode(_extract_quantityDenominator#71006), [product#69685, plant#69686, orderDateTime#69687], true, [quantityDenominator#69726]
(159) Project
Output [3]: [product#69685, plant#69686, orderDateTime#69687]
Input [4]: [product#69685, plant#69686, orderDateTime#69687, quantityDenominator#69726]
(160) BroadcastHashJoin [codegen id : 96]
Left keys [2]: [plantUuid#69677, productUuid#69679]
Right keys [2]: [plant#69686, product#69685]
Join type: Inner
Join condition: (calculationDateTime#69673 <= orderDateTime#69687)
(161) Project [codegen id : 96]
Output [4]: [plantUuid#69677 AS plant#69729, productUuid#69679 AS product#69730, storageLocId#69680, calculationDateTime#69673]
Input [7]: [plantUuid#69677, productUuid#69679, calculationDateTime#69673, storageLocId#69680, product#69685, plant#69686, orderDateTime#69687]
(162) HashAggregate [codegen id : 96]
Input [4]: [plant#69729, product#69730, storageLocId#69680, calculationDateTime#69673]
Keys [3]: [plant#69729, product#69730, storageLocId#69680]
Functions [1]: [partial_max(calculationDateTime#69673)]
Aggregate Attributes [1]: [max#73916]
Results [4]: [plant#69729, product#69730, storageLocId#69680, max#73917]
(163) HashAggregate [codegen id : 96]
Input [4]: [plant#69729, product#69730, storageLocId#69680, max#73917]
Keys [3]: [plant#69729, product#69730, storageLocId#69680]
Functions [1]: [max(calculationDateTime#69673)]
Aggregate Attributes [1]: [max(calculationDateTime#69673)#69456]
Results [4]: [plant#69729, product#69730, storageLocId#69680, max(calculationDateTime#69673)#69456 AS max_calc_datetime#69450]
(164) Exchange
Input [4]: [plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
Arguments: hashpartitioning(coalesce(plant#69729, ), isnull(plant#69729), coalesce(product#69730, ), isnull(product#69730), coalesce(storageLocId#69680, ), isnull(storageLocId#69680), coalesce(max_calc_datetime#69450, 1970-01-01 00:00:00), isnull(max_calc_datetime#69450), 25), ENSURE_REQUIREMENTS, [plan_id=122774]
(165) ShuffleQueryStage
Output [4]: [plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
Arguments: 175
(166) AQEShuffleRead
Input [4]: [plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
Arguments: local
(167) BroadcastHashJoin [codegen id : 99]
Left keys [8]: [coalesce(plant#69448, ), isnull(plant#69448), coalesce(product#69449, ), isnull(product#69449), coalesce(storageLocId#69373, ), isnull(storageLocId#69373), coalesce(calculationDateTime#69308, 1970-01-01 00:00:00), isnull(calculationDateTime#69308)]
Right keys [8]: [coalesce(plant#69729, ), isnull(plant#69729), coalesce(product#69730, ), isnull(product#69730), coalesce(storageLocId#69680, ), isnull(storageLocId#69680), coalesce(max_calc_datetime#69450, 1970-01-01 00:00:00), isnull(max_calc_datetime#69450)]
Join type: Inner
Join condition: None
(168) Project [codegen id : 99]
Output [3]: [plant#69448, product#69449, calculationDateTime#69308]
Input [8]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308, plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
(169) HashAggregate [codegen id : 99]
Input [3]: [plant#69448, product#69449, calculationDateTime#69308]
Keys [2]: [product#69449, plant#69448]
Functions [1]: [partial_min(calculationDateTime#69308)]
Aggregate Attributes [1]: [min#73906]
Results [3]: [product#69449, plant#69448, min#73907]
(170) Exchange
Input [3]: [product#69449, plant#69448, min#73907]
Arguments: hashpartitioning(product#69449, plant#69448, 25), ENSURE_REQUIREMENTS, [plan_id=126301]
(171) ShuffleQueryStage
Output [3]: [product#69449, plant#69448, min#73907]
Arguments: 180
(172) AQEShuffleRead
Input [3]: [product#69449, plant#69448, min#73907]
Arguments: coalesced
(173) HashAggregate [codegen id : 102]
Input [3]: [product#69449, plant#69448, min#73907]
Keys [2]: [product#69449, plant#69448]
Functions [1]: [min(calculationDateTime#69308)]
Aggregate Attributes [1]: [min(calculationDateTime#69308)#69768]
Results [3]: [product#69449, plant#69448, min(calculationDateTime#69308)#69768 AS minCalculationDateTime#69754]
(174) BroadcastExchange
Input [3]: [product#69449, plant#69448, minCalculationDateTime#69754]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=127737]
(175) BroadcastQueryStage
Output [3]: [product#69449, plant#69448, minCalculationDateTime#69754]
Arguments: 183
(176) BroadcastHashJoin [codegen id : 107]
Left keys [2]: [plant#69016, product#69015]
Right keys [2]: [plant#69448, product#69449]
Join type: LeftOuter
Join condition: None
(177) Project [codegen id : 107]
Output [4]: [product#69015, plant#69016, timezonecode#69080, CASE WHEN isnotnull(minCalculationDateTime#69754) THEN minCalculationDateTime#69754 ELSE orderDateTime#69017 END AS stockDatetime#69867]
Input [7]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, product#69449, plant#69448, minCalculationDateTime#69754]
(178) Exchange
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: hashpartitioning(coalesce(plant#69016, ), isnull(plant#69016), coalesce(product#69015, ), isnull(product#69015), 25), ENSURE_REQUIREMENTS, [plan_id=128760]
(179) ShuffleQueryStage
Output [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: 185
(180) AQEShuffleRead
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: local
(181) Scan parquet
Output [3]: [product#70356, plant#70357, orderDateTime#70358]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(182) ColumnarToRow [codegen id : 25]
Input [3]: [product#70356, plant#70357, orderDateTime#70358]
(183) Exchange
Input [3]: [product#70356, plant#70357, orderDateTime#70358]
Arguments: hashpartitioning(product#70356, plant#70357, 25), REPARTITION_BY_NUM, [plan_id=46130]
(184) ShuffleQueryStage
Output [3]: [product#70356, plant#70357, orderDateTime#70358]
Arguments: 39
(185) Scan parquet
Output [8]: [internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367, systemId#70360]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>
(186) ColumnarToRow [codegen id : 26]
Input [8]: [internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367, systemId#70360]
(187) Project [codegen id : 26]
Output [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
Input [8]: [internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367, systemId#70360]
(188) Exchange
Input [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=46149]
(189) ShuffleQueryStage
Output [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
Arguments: 40
(190) Project [codegen id : 71]
Output [1]: [struct(systemId, systemId#70360, internalUUID, internalUUID#70361, plant, plant#70362, product, product#70363, blockedForReplenishmentStartingFrom, blockedForReplenishmentStartingFrom#70364, productionAspect, productionAspect#70365, salesPlant, salesPlant#70366, listing, listing#70367) AS productPlant#70368]
Input [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
(191) BroadcastExchange
Input [1]: [productPlant#70368]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID), coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID)),false), [plan_id=62270]
(192) BroadcastQueryStage
Output [1]: [productPlant#70368]
Arguments: 146
(193) BroadcastHashJoin [codegen id : 103]
Left keys [4]: [coalesce(product#70356, ), isnull(product#70356), coalesce(plant#70357, ), isnull(plant#70357)]
Right keys [4]: [coalesce(productPlant#70368.product.internalRefUUID, ), isnull(productPlant#70368.product.internalRefUUID), coalesce(productPlant#70368.plant.internalRefUUID, ), isnull(productPlant#70368.plant.internalRefUUID)]
Join type: LeftOuter
Join condition: None
(194) ReusedExchange [Reuses operator id: 27]
Output [2]: [internalUUID#70370, timezonecode#70381]
(195) BroadcastQueryStage
Output [2]: [internalUUID#70370, timezonecode#70381]
Arguments: 109
(196) BroadcastHashJoin [codegen id : 103]
Left keys [2]: [coalesce(plant#70357, ), isnull(plant#70357)]
Right keys [2]: [coalesce(internalUUID#70370, ), isnull(internalUUID#70370)]
Join type: LeftOuter
Join condition: None
(197) Project [codegen id : 103]
Output [5]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, CASE WHEN isnotnull(timezonecode#70381) THEN timezonecode#70381 ELSE UTC END AS timezonecode#70382]
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, internalUUID#70370, timezonecode#70381]
(198) ReusedExchange [Reuses operator id: 37]
Output [3]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
(199) BroadcastQueryStage
Output [3]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
Arguments: 102
(200) BroadcastHashJoin [codegen id : 103]
Left keys [1]: [product#70356]
Right keys [1]: [internalUUID#70384]
Join type: LeftOuter
Join condition: None
(201) Project
Output [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, filter(unitOfMeasureConversions#70390, lambdafunction((((lambda u#69110.measurementUnit1.code = productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.code) AND (lambda u#69110.measurementUnit1.internalRefUUID = productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70388.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70388.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70396]
Input [8]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
(202) Project [codegen id : 104]
Output [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, filteredUnitOfMeasureConversions#70396.quantityDenominator AS _extract_quantityDenominator#71131, filteredUnitOfMeasureConversions#70396.quantityNumerator AS _extract_quantityNumerator#71132]
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, filteredUnitOfMeasureConversions#70396]
(203) Generate [codegen id : 104]
Input [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, _extract_quantityDenominator#71131, _extract_quantityNumerator#71132]
Arguments: explode(_extract_quantityNumerator#71132), [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, _extract_quantityDenominator#71131], true, [quantityNumerator#70398]
(204) Generate [codegen id : 104]
Input [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, _extract_quantityDenominator#71131, quantityNumerator#70398]
Arguments: explode(_extract_quantityDenominator#71131), [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, quantityNumerator#70398], true, [quantityDenominator#70399]
(205) Project [codegen id : 104]
Output [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, (cast(quantityNumerator#70398 as double) / cast(quantityDenominator#70399 as double)) AS outboundUnit#70400]
Input [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, quantityNumerator#70398, quantityDenominator#70399]
(206) Project [codegen id : 104]
Output [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, CASE WHEN (isnull(outboundUnit#70400) OR (outboundUnit#70400 <= 0.0)) THEN 1.0 ELSE outboundUnit#70400 END AS outboundUnit#70401]
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70400]
(207) ReusedExchange [Reuses operator id: 51]
Output [6]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407]
(208) BroadcastQueryStage
Output [6]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407]
Arguments: 150
(209) ReusedExchange [Reuses operator id: 64]
Output [3]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
(210) BroadcastQueryStage
Output [3]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
Arguments: 132
(211) ReusedExchange [Reuses operator id: 93]
Output [1]: [plant#70438]
(212) ShuffleQueryStage
Output [1]: [plant#70438]
Arguments: 156
(213) AQEShuffleRead
Input [1]: [plant#70438]
Arguments: coalesced
(214) HashAggregate
Input [1]: [plant#70438]
Keys [1]: [plant#70438]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70438]
(215) BroadcastHashJoin
Left keys [1]: [plantUuid#70433]
Right keys [1]: [plant#70438]
Join type: Inner
Join condition: None
(216) Project
Output [3]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
Input [4]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435, plant#70438]
(217) BroadcastHashJoin [codegen id : 87]
Left keys [2]: [coalesce(plant#70405.internalRefUUID, ), isnull(plant#70405.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#70433, ), isnull(plantUuid#70433)]
Join type: Inner
Join condition: None
(218) Generate
Input [9]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407, plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
Arguments: explode(filter(stockQuantities#70407, lambdafunction(((lambda x#69270.storageLocation.internalRefUuid = storageLocUuid#70435) AND (lambda x#69270.storageLocation.Id = storageLocId#70434)), lambda x#69270, false))), [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, plantUuid#70433], false, [stockQuantity#70498]
(219) ObjectHashAggregate
Input [7]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, plantUuid#70433, stockQuantity#70498]
Keys [5]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402]
Functions [2]: [partial_first(calculationDateTime#70406, false), partial_collect_list(stockQuantity#70498, 0, 0)]
Aggregate Attributes [3]: [first#73946, valueSet#73947, buf#73948]
Results [8]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402, first#73949, valueSet#73950, buf#73951]
(220) ObjectHashAggregate
Input [8]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402, first#73949, valueSet#73950, buf#73951]
Keys [5]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402]
Functions [2]: [first(calculationDateTime#70406, false), collect_list(stockQuantity#70498, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#70406)()#69321, collect_list(stockQuantity#70498, 0, 0)#69322]
Results [4]: [product#70404, plant#70405, first(calculationDateTime#70406)()#69321 AS calculationDateTime#70499, collect_list(stockQuantity#70498, 0, 0)#69322 AS stockQuantities#70500]
(221) Filter [codegen id : 88]
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantities#70500]
Condition : ((size(stockQuantities#70500, true) > 0) AND isnotnull(calculationDateTime#70499))
(222) Generate [codegen id : 88]
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantities#70500]
Arguments: explode(stockQuantities#70500), [product#70404, plant#70405, calculationDateTime#70499], false, [stockQuantity#70501]
(223) Filter [codegen id : 88]
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantity#70501]
Condition : (((isnotnull(stockQuantity#70501.specialStockIndicator.code) AND isnotnull(stockQuantity#70501.stockType.code)) AND (stockQuantity#70501.specialStockIndicator.code = )) AND (stockQuantity#70501.stockType.code = 01))
(224) Project [codegen id : 88]
Output [5]: [plant#70405.internalRefUUID AS plantUuid#70503, product#70404.internalRefUUID AS productUuid#70505, calculationDateTime#70499, stockQuantity#70501.storagelocation.id AS storageLocId#70506, stockQuantity#70501.quantity.measure AS quantity#70508]
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantity#70501]
(225) Exchange
Input [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: hashpartitioning(productUuid#70505, plantUuid#70503, 25), ENSURE_REQUIREMENTS, [plan_id=114769]
(226) ShuffleQueryStage
Output [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: 164
(227) AQEShuffleRead
Input [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: local
(228) BroadcastExchange
Input [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true], input[1, string, true]),false), [plan_id=120649]
(229) BroadcastQueryStage
Output [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: 167
(230) ReusedExchange [Reuses operator id: 115]
Output [3]: [product#70513, plant#70514, orderDateTime#70515]
(231) ShuffleQueryStage
Output [3]: [product#70513, plant#70514, orderDateTime#70515]
Arguments: 58
(232) ReusedExchange [Reuses operator id: 18]
Output [4]: [_extract_code#71399, _extract_internalRefUUID#71400, _extract_internalRefUUID#71402, _extract_internalRefUUID#71401]
(233) BroadcastQueryStage
Output [4]: [_extract_code#71399, _extract_internalRefUUID#71400, _extract_internalRefUUID#71402, _extract_internalRefUUID#71401]
Arguments: 136
(234) BroadcastHashJoin [codegen id : 91]
Left keys [4]: [coalesce(product#70513, ), isnull(product#70513), coalesce(plant#70514, ), isnull(plant#70514)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71402, ), isnull(_extract_internalRefUUID#71402), coalesce(_extract_internalRefUUID#71401, ), isnull(_extract_internalRefUUID#71401)]
Join type: LeftOuter
Join condition: None
(235) Project [codegen id : 91]
Output [5]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400]
Input [7]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400, _extract_internalRefUUID#71402, _extract_internalRefUUID#71401]
(236) ReusedExchange [Reuses operator id: 80]
Output [1]: [internalUUID#70529]
(237) BroadcastQueryStage
Output [1]: [internalUUID#70529]
Arguments: 115
(238) BroadcastHashJoin [codegen id : 91]
Left keys [2]: [coalesce(plant#70514, ), isnull(plant#70514)]
Right keys [2]: [coalesce(internalUUID#70529, ), isnull(internalUUID#70529)]
Join type: LeftOuter
Join condition: None
(239) Project [codegen id : 91]
Output [5]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400]
Input [6]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400, internalUUID#70529]
(240) ReusedExchange [Reuses operator id: 37]
Output [3]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
(241) BroadcastQueryStage
Output [3]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
Arguments: 104
(242) BroadcastHashJoin [codegen id : 91]
Left keys [1]: [product#70513]
Right keys [1]: [internalUUID#70543]
Join type: LeftOuter
Join condition: None
(243) Project
Output [4]: [product#70513, plant#70514, orderDateTime#70515, filter(unitOfMeasureConversions#70549, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71399) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71400)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70547.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70547.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70554]
Input [8]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400, internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
(244) Generate
Input [4]: [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554]
Arguments: explode(filteredUnitOfMeasureConversions#70554.quantityNumerator), [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554], true, [quantityNumerator#70555]
(245) Project
Output [4]: [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554.quantityDenominator AS _extract_quantityDenominator#71032]
Input [5]: [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554, quantityNumerator#70555]
(246) Generate
Input [4]: [product#70513, plant#70514, orderDateTime#70515, _extract_quantityDenominator#71032]
Arguments: explode(_extract_quantityDenominator#71032), [product#70513, plant#70514, orderDateTime#70515], true, [quantityDenominator#70556]
(247) Project
Output [3]: [product#70513, plant#70514, orderDateTime#70515]
Input [4]: [product#70513, plant#70514, orderDateTime#70515, quantityDenominator#70556]
(248) BroadcastHashJoin [codegen id : 92]
Left keys [2]: [plantUuid#70503, productUuid#70505]
Right keys [2]: [plant#70514, product#70513]
Join type: Inner
Join condition: (calculationDateTime#70499 <= orderDateTime#70515)
(249) Project [codegen id : 92]
Output [5]: [plantUuid#70503 AS plant#70559, productUuid#70505 AS product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Input [8]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508, product#70513, plant#70514, orderDateTime#70515]
(250) Exchange
Input [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: hashpartitioning(coalesce(plant#70559, ), isnull(plant#70559), coalesce(product#70560, ), isnull(product#70560), coalesce(storageLocId#70506, ), isnull(storageLocId#70506), coalesce(calculationDateTime#70499, 1970-01-01 00:00:00), isnull(calculationDateTime#70499), 25), ENSURE_REQUIREMENTS, [plan_id=122067]
(251) ShuffleQueryStage
Output [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: 173
(252) AQEShuffleRead
Input [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: local
(253) BroadcastExchange
Input [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true]), coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, timestamp, true], 1970-01-01 00:00:00), isnull(input[3, timestamp, true])),false), [plan_id=124975]
(254) BroadcastQueryStage
Output [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: 178
(255) ReusedExchange [Reuses operator id: 164]
Output [4]: [plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
(256) ShuffleQueryStage
Output [4]: [plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
Arguments: 177
(257) AQEShuffleRead
Input [4]: [plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
Arguments: local
(258) BroadcastHashJoin [codegen id : 100]
Left keys [8]: [coalesce(plant#70559, ), isnull(plant#70559), coalesce(product#70560, ), isnull(product#70560), coalesce(storageLocId#70506, ), isnull(storageLocId#70506), coalesce(calculationDateTime#70499, 1970-01-01 00:00:00), isnull(calculationDateTime#70499)]
Right keys [8]: [coalesce(plant#70696, ), isnull(plant#70696), coalesce(product#70697, ), isnull(product#70697), coalesce(storageLocId#70647, ), isnull(storageLocId#70647), coalesce(max_calc_datetime#70698, 1970-01-01 00:00:00), isnull(max_calc_datetime#70698)]
Join type: Inner
Join condition: None
(259) Project [codegen id : 100]
Output [4]: [plant#70559, product#70560, calculationDateTime#70499, quantity#70508]
Input [9]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508, plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
(260) HashAggregate [codegen id : 100]
Input [4]: [plant#70559, product#70560, calculationDateTime#70499, quantity#70508]
Keys [2]: [product#70560, plant#70559]
Functions [2]: [partial_min(calculationDateTime#70499), partial_sum(quantity#70508)]
Aggregate Attributes [2]: [min#73942, sum#73943]
Results [4]: [product#70560, plant#70559, min#73944, sum#73945]
(261) Exchange
Input [4]: [product#70560, plant#70559, min#73944, sum#73945]
Arguments: hashpartitioning(product#70560, plant#70559, 25), ENSURE_REQUIREMENTS, [plan_id=126336]
(262) ShuffleQueryStage
Output [4]: [product#70560, plant#70559, min#73944, sum#73945]
Arguments: 181
(263) AQEShuffleRead
Input [4]: [product#70560, plant#70559, min#73944, sum#73945]
Arguments: coalesced
(264) HashAggregate [codegen id : 101]
Input [4]: [product#70560, plant#70559, min#73944, sum#73945]
Keys [2]: [product#70560, plant#70559]
Functions [2]: [min(calculationDateTime#70499), sum(quantity#70508)]
Aggregate Attributes [2]: [min(calculationDateTime#70499)#69768, sum(quantity#70508)#69767]
Results [4]: [product#70560, plant#70559, min(calculationDateTime#70499)#69768 AS minCalculationDateTime#70700, round(sum(quantity#70508)#69767, 0) AS roundedQuantitySum#70701]
(265) BroadcastExchange
Input [4]: [product#70560, plant#70559, minCalculationDateTime#70700, roundedQuantitySum#70701]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=127464]
(266) BroadcastQueryStage
Output [4]: [product#70560, plant#70559, minCalculationDateTime#70700, roundedQuantitySum#70701]
Arguments: 182
(267) BroadcastHashJoin [codegen id : 104]
Left keys [2]: [plant#70357, product#70356]
Right keys [2]: [plant#70559, product#70560]
Join type: LeftOuter
Join condition: None
(268) Project [codegen id : 104]
Output [8]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, CASE WHEN isnotnull(minCalculationDateTime#70700) THEN minCalculationDateTime#70700 ELSE orderDateTime#70358 END AS stockDatetime#70702, CASE WHEN isnotnull(roundedQuantitySum#70701) THEN array(roundedQuantitySum#70701) ELSE [0.0] END AS stockBatchQuantities#70703]
Input [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, product#70560, plant#70559, minCalculationDateTime#70700, roundedQuantitySum#70701]
(269) Scan parquet
Output [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-demand-service/out/demand/10000002107/0_1_10000002107]
PushedFilters: [IsNotNull(product), IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string,demandChannel:string,demandStream:string,considerVariance:boolean,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,demandQuantity:double,demandVariance:double>>,demandPointInTimeStart:timestamp,demandPointInTimeEnd:timestamp,demandPointInTime:array<struct<demandDateTime:timestamp,demandQuantity:double,demandVariance:double>>>
(270) ColumnarToRow [codegen id : 49]
Input [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
(271) Filter [codegen id : 49]
Input [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
Condition : (isnotnull(product#69898) AND isnotnull(plant#69899))
(272) BroadcastExchange
Input [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
Arguments: HashedRelationBroadcastMode(List(input[0, string, false], input[1, string, false]),false), [plan_id=46806]
(273) BroadcastQueryStage
Output [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
Arguments: 85
(274) BroadcastHashJoin [codegen id : 104]
Left keys [2]: [product#70356, plant#70357]
Right keys [2]: [product#69898, plant#69899]
Join type: LeftOuter
Join condition: None
(275) Project [codegen id : 104]
Output [9]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, CASE WHEN isnotnull(product#69898) THEN struct(demandChannel, demandChannel#69900, demandStream, demandStream#69901, considerVariance, considerVariance#69902, demandTimeBuckets, demandTimeBuckets#69903, demandPointInTimeStart, demandPointInTimeStart#69904, demandPointInTimeEnd, demandPointInTimeEnd#69905, demandPointInTime, demandPointInTime#69906) END AS rawDemands#69977]
Input [17]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
(276) ObjectHashAggregate
Input [9]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, rawDemands#69977]
Keys [9]: [product#70356, plant#70357, orderDateTime#70358, knownfloatingpointnormalized(if (isnull(productPlant#70368)) null else named_struct(systemId, productPlant#70368.systemId, internalUUID, productPlant#70368.internalUUID, plant, productPlant#70368.plant, product, productPlant#70368.product, blockedForReplenishmentStartingFrom, productPlant#70368.blockedForReplenishmentStartingFrom, productionAspect, knownfloatingpointnormalized(if (isnull(productPlant#70368.productionAspect)) null else named_struct(productMovementPlants, productPlant#70368.productionAspect.productMovementPlants, productPlanningPlants, knownfloatingpointnormalized(if (isnull(productPlant#70368.productionAspect.productPlanningPlants)) null else named_struct(mrpType, productPlant#70368.productionAspect.productPlanningPlants.mrpType, abcIndicator, productPlant#70368.productionAspect.productPlanningPlants.abcIndicator, mrpController, productPlant#70368.productionAspect.productPlanningPlants.mrpController, minimumLotSize, knownfloatingpointnormalized(normalizenanandzero(productPlant#70368.productionAspect.productPlanningPlants.minimumLotSize)), minimumSafetyStock, knownfloatingpointnormalized(normalizenanandzero(productPlant#70368.productionAspect.productPlanningPlants.minimumSafetyStock)), logisticalRoundingProfile, productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile, sourceOfSupplyCategory, productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory)))), salesPlant, productPlant#70368.salesPlant, listing, productPlant#70368.listing)) AS productPlant#70368, timezonecode#70382, knownfloatingpointnormalized(normalizenanandzero(outboundUnit#70401)) AS outboundUnit#70401, stockDatetime#70702, knownfloatingpointnormalized(transform(stockBatchQuantities#70703, lambdafunction(knownfloatingpointnormalized(normalizenanandzero(lambda arg#85171)), lambda arg#85171, false))) AS stockBatchQuantities#70703, [7226582400000000] AS [7226582400000000]#85170]
Functions [1]: [partial_collect_list(rawDemands#69977, 0, 0)]
Aggregate Attributes [1]: [buf#73940]
Results [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000]#85170, buf#73941]
(277) ObjectHashAggregate
Input [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000]#85170, buf#73941]
Keys [9]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000]#85170]
Functions [1]: [collect_list(rawDemands#69977, 0, 0)]
Aggregate Attributes [1]: [collect_list(rawDemands#69977, 0, 0)#70170]
Results [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000] AS stockBatchShelfLifeEndDatetimes#70704, collect_list(rawDemands#69977, 0, 0)#70170 AS rawDemands#70004]
(278) DeserializeToObject [codegen id : 105]
Input [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, stockBatchShelfLifeEndDatetimes#70704, rawDemands#70004]
Arguments: createexternalrow(invoke(product#70356.toString()), invoke(plant#70357.toString()), static_invoke(DateTimeUtils.toJavaTimestamp(orderDateTime#70358)), if (isnull(productPlant#70368)) null else createexternalrow(invoke(productPlant#70368.systemId.toString()), invoke(productPlant#70368.internalUUID.toString()), if (isnull(productPlant#70368.plant)) null else createexternalrow(invoke(productPlant#70368.plant.internalRefUUID.toString()), invoke(productPlant#70368.plant.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), if (isnull(productPlant#70368.product)) null else createexternalrow(invoke(productPlant#70368.product.internalRefUUID.toString()), invoke(productPlant#70368.product.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(DateTimeUtils.toJavaDate(productPlant#70368.blockedForReplenishmentStartingFrom)), if (isnull(productPlant#70368.productionAspect)) null else createexternalrow(if (isnull(productPlant#70368.productionAspect.productMovementPlants)) null else createexternalrow(if (isnull(productPlant#70368.productionAspect.productMovementPlants.unitOfIssue)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants)) null else createexternalrow(if (isnull(productPlant#70368.productionAspect.productPlanningPlants.mrpType)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpType.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpType.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.abcIndicator)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.abcIndicator.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.abcIndicator.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.mrpController)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpController.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpController.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(java.lang.Double.valueOf(productPlant#70368.productionAspect.productPlanningPlants.minimumLotSize)), static_invoke(java.lang.Double.valueOf(productPlant#70368.productionAspect.productPlanningPlants.minimumSafetyStock)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile.logisticalRoundingProfile.toString()), StructField(internalRefUUID,StringType,true), StructField(logisticalRoundingProfile,StringType,true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(minimumLotSize,DoubleType,true), StructField(minimumSafetyStock,DoubleType,true), StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true), StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)), if (isnull(productPlant#70368.salesPlant)) null else createexternalrow(if (isnull(productPlant#70368.salesPlant.loadingGroup)) null else createexternalrow(invoke(productPlant#70368.salesPlant.loadingGroup.internalRefUUID.toString()), invoke(productPlant#70368.salesPlant.loadingGroup.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), mapobjects(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1), if (isnull(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validFrom)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validTo)), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).isListed)), StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true)), productPlant#70368.listing, Some(class scala.collection.mutable.ArraySeq)), StructField(systemId,StringType,true), StructField(internalUUID,StringType,true), StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(blockedForReplenishmentStartingFrom,DateType,true), StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true), StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)), invoke(timezonecode#70382.toString()), static_invoke(java.lang.Double.valueOf(outboundUnit#70401)), static_invoke(DateTimeUtils.toJavaTimestamp(stockDatetime#70702)), mapobjects(lambdavariable(MapObject, DoubleType, true, -2), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, DoubleType, true, -2))), stockBatchQuantities#70703, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, TimestampType, false, -3), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, TimestampType, false, -3))), stockBatchShelfLifeEndDatetimes#70704, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4), createexternalrow(invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandChannel.toString()), invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandStream.toString()), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).considerVariance)), mapobjects(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5), if (isnull(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandStartDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandVariance)), StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandTimeBuckets, Some(class scala.collection.mutable.ArraySeq)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeStart)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeEnd)), mapobjects(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6), if (isnull(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandVariance)), StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTime, Some(class scala.collection.mutable.ArraySeq)), StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)), rawDemands#70004, Some(class scala.collection.mutable.ArraySeq)), StructField(product,StringType,true), StructField(plant,StringType,true), StructField(orderDateTime,TimestampType,true), StructField(productPlant,StructType(StructField(systemId,StringType,true),StructField(internalUUID,StringType,true),StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(blockedForReplenishmentStartingFrom,DateType,true),StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true),StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)),true), StructField(timezonecode,StringType,true), StructField(outboundUnit,DoubleType,true), StructField(stockDatetime,TimestampType,true), StructField(stockBatchQuantities,ArrayType(DoubleType,true),false), StructField(stockBatchShelfLifeEndDatetimes,ArrayType(TimestampType,false),false), StructField(rawDemands,ArrayType(StructType(StructField(demandChannel,StringType,true),StructField(demandStream,StringType,true),StructField(considerVariance,BooleanType,true),StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true),StructField(demandPointInTimeStart,TimestampType,true),StructField(demandPointInTimeEnd,TimestampType,true),StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)),false),false)), obj#70342: org.apache.spark.sql.Row
(279) MapElements [codegen id : 105]
Input [1]: [obj#70342]
Arguments: com.sap.s4hana.eureka.business.crporderquantitykpiservice.core.business.datasnapshot.DemandTransformatorImpl$$Lambda/0x00000008024b0d88@22b451ac, obj#70347: org.apache.spark.sql.Row
(280) SerializeFromObject [codegen id : 105]
Input [1]: [obj#70347]
Arguments: [if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(0))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, product), StringType, ObjectType(class java.lang.String)))) AS product#70352, if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(1))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, plant), StringType, ObjectType(class java.lang.String)))) AS plant#70353]
(281) Exchange
Input [2]: [product#70352, plant#70353]
Arguments: hashpartitioning(coalesce(plant#70353, ), isnull(plant#70353), coalesce(product#70352, ), isnull(product#70352), 25), ENSURE_REQUIREMENTS, [plan_id=128578]
(282) ShuffleQueryStage
Output [2]: [product#70352, plant#70353]
Arguments: 184
(283) AQEShuffleRead
Input [2]: [product#70352, plant#70353]
Arguments: local
(284) BroadcastExchange
Input [2]: [product#70352, plant#70353]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=129190]
(285) BroadcastQueryStage
Output [2]: [product#70352, plant#70353]
Arguments: 186
(286) BroadcastHashJoin [codegen id : 108]
Left keys [4]: [coalesce(plant#69016, ), isnull(plant#69016), coalesce(product#69015, ), isnull(product#69015)]
Right keys [4]: [coalesce(plant#70353, ), isnull(plant#70353), coalesce(product#70352, ), isnull(product#70352)]
Join type: LeftOuter
Join condition: None
(287) Project [codegen id : 108]
Output [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Input [6]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867, product#70352, plant#70353]
(288) Exchange
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: hashpartitioning(plant#69016, 25), ENSURE_REQUIREMENTS, [plan_id=129261]
(289) ShuffleQueryStage
Output [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: 187
(290) AQEShuffleRead
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: local
(291) BroadcastExchange
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true]),false), [plan_id=129311]
(292) BroadcastQueryStage
Output [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: 188
(293) ReusedExchange [Reuses operator id: 57]
Output [1]: [distributionCenters#70837]
(294) ShuffleQueryStage
Output [1]: [distributionCenters#70837]
Arguments: 87
(295) Generate [codegen id : 69]
Input [1]: [distributionCenters#70837]
Arguments: explode(distributionCenters#70837), false, [distributionCenters#70853]
(296) Filter [codegen id : 69]
Input [1]: [distributionCenters#70853]
Condition : (((size(distributionCenters#70853.storageLocations.internalRefUUID, true) > 0) AND isnotnull(distributionCenters#70853.storageLocations.internalRefUUID)) AND isnotnull(distributionCenters#70853.internalRefUUID))
(297) Project [codegen id : 69]
Output [2]: [distributionCenters#70853.internalRefUUID AS _extract_internalRefUUID#71059, distributionCenters#70853.storageLocations.internalRefUUID AS _extract_internalRefUUID#71430]
Input [1]: [distributionCenters#70853]
(298) Generate [codegen id : 69]
Input [2]: [_extract_internalRefUUID#71059, _extract_internalRefUUID#71430]
Arguments: explode(_extract_internalRefUUID#71430), [_extract_internalRefUUID#71059], false, [storageLocations#70856]
(299) Filter [codegen id : 69]
Input [2]: [_extract_internalRefUUID#71059, storageLocations#70856]
Condition : isnotnull(storageLocations#70856)
(300) Project [codegen id : 69]
Output [2]: [_extract_internalRefUUID#71059 AS plantUuid#70860, storageLocations#70856 AS storageLocUuid#70864]
Input [2]: [_extract_internalRefUUID#71059, storageLocations#70856]
(301) BroadcastExchange
Input [2]: [plantUuid#70860, storageLocUuid#70864]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=60900]
(302) BroadcastQueryStage
Output [2]: [plantUuid#70860, storageLocUuid#70864]
Arguments: 143
(303) ReusedExchange [Reuses operator id: 93]
Output [1]: [plant#70870]
(304) ShuffleQueryStage
Output [1]: [plant#70870]
Arguments: 160
(305) HashAggregate
Input [1]: [plant#70870]
Keys [1]: [plant#70870]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70870]
(306) BroadcastHashJoin
Left keys [1]: [plantUuid#70860]
Right keys [1]: [plant#70870]
Join type: Inner
Join condition: None
(307) Project
Output [2]: [plantUuid#70860 AS plant#70808, storageLocUuid#70864]
Input [3]: [plantUuid#70860, storageLocUuid#70864, plant#70870]
(308) BroadcastHashJoin [codegen id : 109]
Left keys [1]: [plant#69016]
Right keys [1]: [plant#70808]
Join type: Inner
Join condition: None
(309) Project [codegen id : 109]
Output [5]: [plant#69016, product#69015, timezonecode#69080, stockDatetime#69867, storageLocUuid#70864]
Input [6]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867, plant#70808, storageLocUuid#70864]
(310) Scan parquet
Output [6]: [internalUUID#70739, id#70740, supplier#70742, isMarkedForDeletion#70745, items#70746, systemId#70738]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-procurement-purchaseorder]
PushedFilters: [IsNotNull(isMarkedForDeletion), Not(EqualTo(isMarkedForDeletion,true)), IsNotNull(items)]
ReadSchema: struct<internalUUID:string,id:string,supplier:struct<internalRefUUID:string,id:string>,isMarkedForDeletion:boolean,items:array<struct<number:string,completeIndicator:boolean,returnIndicator:boolean,deletionIndicator:boolean,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:double,quantityUnit:struct<internalRefUUID:string,code:string>,quantityNumerator:double,quantityDenominator:double,scheduleLines:array<struct<number:string,deliveryAt:timestamp,quantity:double,quantityUnit:struct<internalRefUUID:string,code:string>,receivedQuantity:double,deliveredQuantity:double,productAvailabilityDateTime:timestamp>>,confirmations:array<struct<number:string,confirmationCategory:struct<internalRefUUID:string,code:string>,confirmationDeliveryDate:date,confirmedDeliveryAt:timestamp,confirmedQuantity:double,mrpRelevantQuantity:double,confirmationCreationCategory:string,deletionIndicator:boolean,isRelevantForConfirmation:boolean,delivery:struct<internalRefUUID:string,id:string>,deliveryItemNumber:string>>>>>
(311) ColumnarToRow [codegen id : 55]
Input [6]: [internalUUID#70739, id#70740, supplier#70742, isMarkedForDeletion#70745, items#70746, systemId#70738]
(312) Filter [codegen id : 55]
Input [6]: [internalUUID#70739, id#70740, supplier#70742, isMarkedForDeletion#70745, items#70746, systemId#70738]
Condition : (((isnotnull(isMarkedForDeletion#70745) AND NOT isMarkedForDeletion#70745) AND (size(items#70746, true) > 0)) AND isnotnull(items#70746))
(313) Project [codegen id : 55]
Output [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Input [6]: [internalUUID#70739, id#70740, supplier#70742, isMarkedForDeletion#70745, items#70746, systemId#70738]
(314) Exchange
Input [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=47082]
(315) ShuffleQueryStage
Output [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Arguments: 96
(316) Generate [codegen id : 72]
Input [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Arguments: explode(items#70746), [internalUUID#70739, id#70740, supplier#70742], false, [item#70754]
(317) Filter [codegen id : 72]
Input [4]: [internalUUID#70739, id#70740, supplier#70742, item#70754]
Condition : ((((isnotnull(item#70754.deletionIndicator) AND isnotnull(item#70754.completeIndicator)) AND isnotnull(item#70754.returnIndicator)) AND ((NOT item#70754.deletionIndicator AND NOT item#70754.completeIndicator) AND NOT item#70754.returnIndicator)) AND ((isnotnull(item#70754.product.internalRefUUID) AND isnotnull(item#70754.plant.internalRefUUID)) AND isnotnull(item#70754.storageLocation.internalRefUUID)))
(318) Project [codegen id : 72]
Output [14]: [id#70740, internalUUID#70739, supplier#70742.internalRefUUID AS supplierUuid#70749, item#70754.plant.internalRefUUID AS plant#70763, item#70754.product.internalRefUUID AS product#70764, item#70754.plant.id AS externalPlantId#70765, item#70754.product.id AS externalProductId#70766, item#70754.storageLocation.internalRefUUID AS storageLocUuid#70767, item#70754.quantityUnit.code AS quantityUom#70768, item#70754.quantityUnit.internalRefUUID AS quantityUomUuid#70769, item#70754.quantityNumerator AS quantityNumerator#70771, item#70754.quantityDenominator AS quantityDenominator#70772, item#70754.scheduleLines AS scheduleLines#70774, item#70754.confirmations AS confirmations#70776]
Input [4]: [internalUUID#70739, id#70740, supplier#70742, item#70754]
(319) BroadcastExchange
Input [14]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776]
Arguments: HashedRelationBroadcastMode(List(input[4, string, true], input[3, string, true], input[7, string, true]),false), [plan_id=63658]
(320) BroadcastQueryStage
Output [14]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776]
Arguments: 147
(321) BroadcastHashJoin [codegen id : 109]
Left keys [3]: [product#69015, plant#69016, storageLocUuid#70864]
Right keys [3]: [product#70764, plant#70763, storageLocUuid#70767]
Join type: Inner
Join condition: None
(322) Project
Output [17]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776, stockDatetime#69867, timezonecode#69080, filter(confirmations#70776, lambdafunction(((NOT lambda x_1#70965.deletionIndicator AND lambda x_1#70965.isRelevantForConfirmation) AND isnotnull(lambda x_1#70965.confirmedDeliveryAt)), lambda x_1#70965, false)) AS validConfirmations#70964]
Input [19]: [plant#69016, product#69015, timezonecode#69080, stockDatetime#69867, storageLocUuid#70864, id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776]
(323) ResultQueryStage
Output [17]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776, stockDatetime#69867, timezonecode#69080, validConfirmations#70964]
Arguments: 189
(324) Filter
Input [3]: [product#69015, plant#69016, orderDateTime#69017]
Condition : (isnotnull(plant#69016) AND isnotnull(product#69015))
(325) Exchange
Input [3]: [product#69015, plant#69016, orderDateTime#69017]
Arguments: hashpartitioning(product#69015, plant#69016, 25), REPARTITION_BY_NUM, [plan_id=31626]
(326) Project
Output [4]: [plant#69030.internalRefUUID AS _extract_internalRefUUID#71356, product#69032.internalRefUUID AS _extract_internalRefUUID#71355, productionAspect#69035.productMovementPlants.unitOfIssue.code AS _extract_code#71358, productionAspect#69035.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71359]
Input [4]: [plant#69030, product#69032, productionAspect#69035, systemId#69026]
(327) Exchange
Input [4]: [_extract_internalRefUUID#71356, _extract_internalRefUUID#71355, _extract_code#71358, _extract_internalRefUUID#71359]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30733]
(328) Project
Output [4]: [_extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
Input [4]: [_extract_internalRefUUID#71356, _extract_internalRefUUID#71355, _extract_code#71358, _extract_internalRefUUID#71359]
(329) BroadcastExchange
Input [4]: [_extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32025]
(330) BroadcastHashJoin
Left keys [4]: [coalesce(product#69015, ), isnull(product#69015), coalesce(plant#69016, ), isnull(plant#69016)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71355, ), isnull(_extract_internalRefUUID#71355), coalesce(_extract_internalRefUUID#71356, ), isnull(_extract_internalRefUUID#71356)]
Join type: LeftOuter
Join condition: None
(331) Project
Output [5]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359]
Input [7]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, _extract_internalRefUUID#71355, _extract_internalRefUUID#71356]
(332) Project
Output [2]: [internalUUID#69050, address#69056.timeZone.code AS timezonecode#69073]
Input [3]: [internalUUID#69050, address#69056, systemId#69049]
(333) Exchange
Input [2]: [internalUUID#69050, timezonecode#69073]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30739]
(334) BroadcastExchange
Input [2]: [internalUUID#69050, timezonecode#69073]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32032]
(335) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69016, ), isnull(plant#69016)]
Right keys [2]: [coalesce(internalUUID#69050, ), isnull(internalUUID#69050)]
Join type: LeftOuter
Join condition: None
(336) Project
Output [6]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, CASE WHEN isnotnull(timezonecode#69073) THEN timezonecode#69073 ELSE UTC END AS timezonecode#69080]
Input [7]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, internalUUID#69050, timezonecode#69073]
(337) Filter
Input [4]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088, systemId#69081]
Condition : isnotnull(internalUUID#69082)
(338) Project
Output [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Input [4]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088, systemId#69081]
(339) Exchange
Input [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30749]
(340) BroadcastExchange
Input [3]: [internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32040]
(341) BroadcastHashJoin
Left keys [1]: [product#69015]
Right keys [1]: [internalUUID#69082]
Join type: LeftOuter
Join condition: None
(342) Project
Output [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filter(unitOfMeasureConversions#69088, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71358) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71359)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69086.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69086.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69108]
Input [9]: [product#69015, plant#69016, orderDateTime#69017, _extract_code#71358, _extract_internalRefUUID#71359, timezonecode#69080, internalUUID#69082, baseUnitOfMeasure#69086, unitOfMeasureConversions#69088]
(343) Generate
Input [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108]
Arguments: explode(filteredUnitOfMeasureConversions#69108.quantityNumerator), [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108], true, [quantityNumerator#69123]
(344) Project
Output [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108.quantityDenominator AS _extract_quantityDenominator#70967]
Input [6]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, filteredUnitOfMeasureConversions#69108, quantityNumerator#69123]
(345) Generate
Input [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, _extract_quantityDenominator#70967]
Arguments: explode(_extract_quantityDenominator#70967), [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080], true, [quantityDenominator#69130]
(346) Project
Output [4]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080]
Input [5]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, quantityDenominator#69130]
(347) Sort
Input [4]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080]
Arguments: [product#69015 ASC NULLS FIRST, plant#69016 ASC NULLS FIRST], false, 0
(348) Filter
Input [6]: [internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, systemId#69251]
Condition : (isnotnull(plant#69254.internalRefUUID) AND isnotnull(product#69253.internalRefUUID))
(349) Project
Output [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Input [6]: [internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, systemId#69251]
(350) Exchange
Input [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30761]
(351) BroadcastExchange
Input [6]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32102]
(352) Filter
Input [2]: [marketUnit#69147, distributionCenters#69161]
Condition : (((marketUnit#69147 <=> Daily_grocery) AND (size(distributionCenters#69161, true) > 0)) AND isnotnull(distributionCenters#69161))
(353) Project
Output [1]: [distributionCenters#69161]
Input [2]: [marketUnit#69147, distributionCenters#69161]
(354) Exchange
Input [1]: [distributionCenters#69161]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30765]
(355) Generate
Input [1]: [distributionCenters#69161]
Arguments: explode(distributionCenters#69161), false, [distributionCenters#69184]
(356) Filter
Input [1]: [distributionCenters#69184]
Condition : (((size(distributionCenters#69184.storageLocations, true) > 0) AND isnotnull(distributionCenters#69184.storageLocations)) AND isnotnull(distributionCenters#69184.internalRefUUID))
(357) Project
Output [2]: [distributionCenters#69184.internalRefUUID AS _extract_internalRefUUID#70975, distributionCenters#69184.storageLocations AS _extract_storageLocations#70976]
Input [1]: [distributionCenters#69184]
(358) Generate
Input [2]: [_extract_internalRefUUID#70975, _extract_storageLocations#70976]
Arguments: explode(_extract_storageLocations#70976), [_extract_internalRefUUID#70975], false, [storageLocations#69198]
(359) Project
Output [3]: [_extract_internalRefUUID#70975 AS plantUuid#69189, storageLocations#69198.id AS storageLocId#69201, storageLocations#69198.internalRefUUID AS storageLocUuid#69202]
Input [2]: [_extract_internalRefUUID#70975, storageLocations#69198]
(360) BroadcastExchange
Input [3]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32091]
(361) Filter
Input [2]: [product#69783, plant#69784]
Condition : isnotnull(plant#69784)
(362) Exchange
Input [2]: [product#69783, plant#69784]
Arguments: hashpartitioning(product#69783, plant#69784, 25), REPARTITION_BY_NUM, [plan_id=31649]
(363) Scan parquet
Output [4]: [plant#69789, product#69790, productionAspect#69792, systemId#69787]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(364) Project
Output [4]: [plant#69789.internalRefUUID AS _extract_internalRefUUID#71366, product#69790.internalRefUUID AS _extract_internalRefUUID#71363, productionAspect#69792.productMovementPlants.unitOfIssue.code AS _extract_code#71364, productionAspect#69792.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71365]
Input [4]: [plant#69789, product#69790, productionAspect#69792, systemId#69787]
(365) Exchange
Input [4]: [_extract_internalRefUUID#71366, _extract_internalRefUUID#71363, _extract_code#71364, _extract_internalRefUUID#71365]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30777]
(366) Project
Output [4]: [_extract_code#71364, _extract_internalRefUUID#71365, _extract_internalRefUUID#71363, _extract_internalRefUUID#71366]
Input [4]: [_extract_internalRefUUID#71366, _extract_internalRefUUID#71363, _extract_code#71364, _extract_internalRefUUID#71365]
(367) BroadcastExchange
Input [4]: [_extract_code#71364, _extract_internalRefUUID#71365, _extract_internalRefUUID#71363, _extract_internalRefUUID#71366]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32054]
(368) BroadcastHashJoin
Left keys [4]: [coalesce(product#69783, ), isnull(product#69783), coalesce(plant#69784, ), isnull(plant#69784)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71363, ), isnull(_extract_internalRefUUID#71363), coalesce(_extract_internalRefUUID#71366, ), isnull(_extract_internalRefUUID#71366)]
Join type: LeftOuter
Join condition: None
(369) Project
Output [4]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365]
Input [6]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365, _extract_internalRefUUID#71363, _extract_internalRefUUID#71366]
(370) Project
Output [1]: [internalUUID#69810]
Input [2]: [internalUUID#69810, systemId#69809]
(371) Exchange
Input [1]: [internalUUID#69810]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30783]
(372) BroadcastExchange
Input [1]: [internalUUID#69810]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32058]
(373) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69784, ), isnull(plant#69784)]
Right keys [2]: [coalesce(internalUUID#69810, ), isnull(internalUUID#69810)]
Join type: LeftOuter
Join condition: None
(374) Project
Output [4]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365]
Input [5]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365, internalUUID#69810]
(375) Scan parquet
Output [4]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844, systemId#69837]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(376) Filter
Input [4]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844, systemId#69837]
Condition : isnotnull(internalUUID#69838)
(377) Project
Output [3]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
Input [4]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844, systemId#69837]
(378) Exchange
Input [3]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30789]
(379) BroadcastExchange
Input [3]: [internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32065]
(380) BroadcastHashJoin
Left keys [1]: [product#69783]
Right keys [1]: [internalUUID#69838]
Join type: LeftOuter
Join condition: None
(381) Project
Output [2]: [plant#69784, filter(unitOfMeasureConversions#69844, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71364) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71365)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69842.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69842.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69850]
Input [7]: [product#69783, plant#69784, _extract_code#71364, _extract_internalRefUUID#71365, internalUUID#69838, baseUnitOfMeasure#69842, unitOfMeasureConversions#69844]
(382) Generate
Input [2]: [plant#69784, filteredUnitOfMeasureConversions#69850]
Arguments: explode(filteredUnitOfMeasureConversions#69850.quantityNumerator), [plant#69784, filteredUnitOfMeasureConversions#69850], true, [quantityNumerator#69852]
(383) Project
Output [2]: [plant#69784, filteredUnitOfMeasureConversions#69850.quantityDenominator AS _extract_quantityDenominator#70979]
Input [3]: [plant#69784, filteredUnitOfMeasureConversions#69850, quantityNumerator#69852]
(384) Generate
Input [2]: [plant#69784, _extract_quantityDenominator#70979]
Arguments: explode(_extract_quantityDenominator#70979), [plant#69784], true, [quantityDenominator#69853]
(385) Project
Output [1]: [plant#69784]
Input [2]: [plant#69784, quantityDenominator#69853]
(386) HashAggregate
Input [1]: [plant#69784]
Keys [1]: [plant#69784]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69784]
(387) Exchange
Input [1]: [plant#69784]
Arguments: hashpartitioning(plant#69784, 25), ENSURE_REQUIREMENTS, [plan_id=32087]
(388) HashAggregate
Input [1]: [plant#69784]
Keys [1]: [plant#69784]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69784]
(389) BroadcastHashJoin
Left keys [1]: [plantUuid#69189]
Right keys [1]: [plant#69784]
Join type: Inner
Join condition: None
(390) Project
Output [3]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Input [4]: [plantUuid#69189, storageLocId#69201, storageLocUuid#69202, plant#69784]
(391) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69254.internalRefUUID, ), isnull(plant#69254.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#69189, ), isnull(plantUuid#69189)]
Join type: Inner
Join condition: None
(392) Generate
Input [9]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, stockQuantities#69256, plantUuid#69189, storageLocId#69201, storageLocUuid#69202]
Arguments: explode(filter(stockQuantities#69256, lambdafunction(((lambda x#69270.storageLocation.internalRefUuid = storageLocUuid#69202) AND (lambda x#69270.storageLocation.Id = storageLocId#69201)), lambda x#69270, false))), [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, plantUuid#69189], false, [stockQuantity#69271]
(393) ObjectHashAggregate
Input [7]: [systemId#69251, internalUUID#69252, product#69253, plant#69254, calculationDateTime#69255, plantUuid#69189, stockQuantity#69271]
Keys [5]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251]
Functions [2]: [partial_first(calculationDateTime#69255, false), partial_collect_list(stockQuantity#69271, 0, 0)]
Aggregate Attributes [3]: [first#73908, valueSet#73909, buf#73910]
Results [8]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251, first#73911, valueSet#73912, buf#73913]
(394) ObjectHashAggregate
Input [8]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251, first#73911, valueSet#73912, buf#73913]
Keys [5]: [plantUuid#69189, internalUUID#69252, plant#69254, product#69253, systemId#69251]
Functions [2]: [first(calculationDateTime#69255, false), collect_list(stockQuantity#69271, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#69255)()#69321, collect_list(stockQuantity#69271, 0, 0)#69322]
Results [4]: [product#69253, plant#69254, first(calculationDateTime#69255)()#69321 AS calculationDateTime#69308, collect_list(stockQuantity#69271, 0, 0)#69322 AS stockQuantities#69309]
(395) Filter
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantities#69309]
Condition : ((size(stockQuantities#69309, true) > 0) AND isnotnull(calculationDateTime#69308))
(396) Generate
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantities#69309]
Arguments: explode(stockQuantities#69309), [product#69253, plant#69254, calculationDateTime#69308], false, [stockQuantity#69367]
(397) Filter
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantity#69367]
Condition : (((isnotnull(stockQuantity#69367.specialStockIndicator.code) AND isnotnull(stockQuantity#69367.stockType.code)) AND (stockQuantity#69367.specialStockIndicator.code = )) AND (stockQuantity#69367.stockType.code = 01))
(398) Project
Output [4]: [plant#69254.internalRefUUID AS plantUuid#69370, product#69253.internalRefUUID AS productUuid#69372, calculationDateTime#69308, stockQuantity#69367.storagelocation.id AS storageLocId#69373]
Input [4]: [product#69253, plant#69254, calculationDateTime#69308, stockQuantity#69367]
(399) Exchange
Input [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: hashpartitioning(productUuid#69372, plantUuid#69370, 25), ENSURE_REQUIREMENTS, [plan_id=32155]
(400) Sort
Input [4]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373]
Arguments: [productUuid#69372 ASC NULLS FIRST, plantUuid#69370 ASC NULLS FIRST], false, 0
(401) Filter
Input [3]: [product#69391, plant#69392, orderDateTime#69393]
Condition : ((isnotnull(orderDateTime#69393) AND isnotnull(plant#69392)) AND isnotnull(product#69391))
(402) Exchange
Input [3]: [product#69391, plant#69392, orderDateTime#69393]
Arguments: hashpartitioning(product#69391, plant#69392, 25), REPARTITION_BY_NUM, [plan_id=31694]
(403) Scan parquet
Output [4]: [plant#69397, product#69398, productionAspect#69400, systemId#69395]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(404) Project
Output [4]: [plant#69397.internalRefUUID AS _extract_internalRefUUID#71374, product#69398.internalRefUUID AS _extract_internalRefUUID#71371, productionAspect#69400.productMovementPlants.unitOfIssue.code AS _extract_code#71372, productionAspect#69400.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71373]
Input [4]: [plant#69397, product#69398, productionAspect#69400, systemId#69395]
(405) Exchange
Input [4]: [_extract_internalRefUUID#71374, _extract_internalRefUUID#71371, _extract_code#71372, _extract_internalRefUUID#71373]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30828]
(406) Project
Output [4]: [_extract_code#71372, _extract_internalRefUUID#71373, _extract_internalRefUUID#71371, _extract_internalRefUUID#71374]
Input [4]: [_extract_internalRefUUID#71374, _extract_internalRefUUID#71371, _extract_code#71372, _extract_internalRefUUID#71373]
(407) BroadcastExchange
Input [4]: [_extract_code#71372, _extract_internalRefUUID#71373, _extract_internalRefUUID#71371, _extract_internalRefUUID#71374]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32120]
(408) BroadcastHashJoin
Left keys [4]: [coalesce(product#69391, ), isnull(product#69391), coalesce(plant#69392, ), isnull(plant#69392)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71371, ), isnull(_extract_internalRefUUID#71371), coalesce(_extract_internalRefUUID#71374, ), isnull(_extract_internalRefUUID#71374)]
Join type: LeftOuter
Join condition: None
(409) Project
Output [5]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373]
Input [7]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373, _extract_internalRefUUID#71371, _extract_internalRefUUID#71374]
(410) Scan parquet
Output [2]: [internalUUID#69405, systemId#69404]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(411) Project
Output [1]: [internalUUID#69405]
Input [2]: [internalUUID#69405, systemId#69404]
(412) Exchange
Input [1]: [internalUUID#69405]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30834]
(413) BroadcastExchange
Input [1]: [internalUUID#69405]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32140]
(414) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69392, ), isnull(plant#69392)]
Right keys [2]: [coalesce(internalUUID#69405, ), isnull(internalUUID#69405)]
Join type: LeftOuter
Join condition: None
(415) Project
Output [5]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373]
Input [6]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373, internalUUID#69405]
(416) Scan parquet
Output [4]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425, systemId#69418]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(417) Filter
Input [4]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425, systemId#69418]
Condition : isnotnull(internalUUID#69419)
(418) Project
Output [3]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
Input [4]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425, systemId#69418]
(419) Exchange
Input [3]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30840]
(420) BroadcastExchange
Input [3]: [internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32144]
(421) BroadcastHashJoin
Left keys [1]: [product#69391]
Right keys [1]: [internalUUID#69419]
Join type: LeftOuter
Join condition: None
(422) Project
Output [4]: [product#69391, plant#69392, orderDateTime#69393, filter(unitOfMeasureConversions#69425, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71372) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71373)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69423.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69423.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69430]
Input [8]: [product#69391, plant#69392, orderDateTime#69393, _extract_code#71372, _extract_internalRefUUID#71373, internalUUID#69419, baseUnitOfMeasure#69423, unitOfMeasureConversions#69425]
(423) Generate
Input [4]: [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430]
Arguments: explode(filteredUnitOfMeasureConversions#69430.quantityNumerator), [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430], true, [quantityNumerator#69431]
(424) Project
Output [4]: [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430.quantityDenominator AS _extract_quantityDenominator#70987]
Input [5]: [product#69391, plant#69392, orderDateTime#69393, filteredUnitOfMeasureConversions#69430, quantityNumerator#69431]
(425) Generate
Input [4]: [product#69391, plant#69392, orderDateTime#69393, _extract_quantityDenominator#70987]
Arguments: explode(_extract_quantityDenominator#70987), [product#69391, plant#69392, orderDateTime#69393], true, [quantityDenominator#69432]
(426) Project
Output [3]: [product#69391, plant#69392, orderDateTime#69393]
Input [4]: [product#69391, plant#69392, orderDateTime#69393, quantityDenominator#69432]
(427) Sort
Input [3]: [product#69391, plant#69392, orderDateTime#69393]
Arguments: [product#69391 ASC NULLS FIRST, plant#69392 ASC NULLS FIRST], false, 0
(428) SortMergeJoin
Left keys [2]: [productUuid#69372, plantUuid#69370]
Right keys [2]: [product#69391, plant#69392]
Join type: Inner
Join condition: (calculationDateTime#69308 <= orderDateTime#69393)
(429) Project
Output [4]: [plantUuid#69370 AS plant#69448, productUuid#69372 AS product#69449, storageLocId#69373, calculationDateTime#69308]
Input [7]: [plantUuid#69370, productUuid#69372, calculationDateTime#69308, storageLocId#69373, product#69391, plant#69392, orderDateTime#69393]
(430) Exchange
Input [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: hashpartitioning(coalesce(plant#69448, ), isnull(plant#69448), coalesce(product#69449, ), isnull(product#69449), coalesce(storageLocId#69373, ), isnull(storageLocId#69373), coalesce(calculationDateTime#69308, 1970-01-01 00:00:00), isnull(calculationDateTime#69308), 25), ENSURE_REQUIREMENTS, [plan_id=32282]
(431) Sort
Input [4]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308]
Arguments: [coalesce(plant#69448, ) ASC NULLS FIRST, isnull(plant#69448) ASC NULLS FIRST, coalesce(product#69449, ) ASC NULLS FIRST, isnull(product#69449) ASC NULLS FIRST, coalesce(storageLocId#69373, ) ASC NULLS FIRST, isnull(storageLocId#69373) ASC NULLS FIRST, coalesce(calculationDateTime#69308, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(calculationDateTime#69308) ASC NULLS FIRST], false, 0
(432) Scan parquet
Output [6]: [internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597, systemId#69592]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(433) Filter
Input [6]: [internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597, systemId#69592]
Condition : (isnotnull(plant#69595.internalRefUUID) AND isnotnull(product#69594.internalRefUUID))
(434) Project
Output [6]: [systemId#69592, internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597]
Input [6]: [internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597, systemId#69592]
(435) Exchange
Input [6]: [systemId#69592, internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30859]
(436) BroadcastExchange
Input [6]: [systemId#69592, internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32213]
(437) Scan parquet
Output [2]: [marketUnit#69598, distributionCenters#69604]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(438) Filter
Input [2]: [marketUnit#69598, distributionCenters#69604]
Condition : (((marketUnit#69598 <=> Daily_grocery) AND (size(distributionCenters#69604, true) > 0)) AND isnotnull(distributionCenters#69604))
(439) Project
Output [1]: [distributionCenters#69604]
Input [2]: [marketUnit#69598, distributionCenters#69604]
(440) Exchange
Input [1]: [distributionCenters#69604]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30863]
(441) Generate
Input [1]: [distributionCenters#69604]
Arguments: explode(distributionCenters#69604), false, [distributionCenters#69622]
(442) Filter
Input [1]: [distributionCenters#69622]
Condition : (((size(distributionCenters#69622.storageLocations, true) > 0) AND isnotnull(distributionCenters#69622.storageLocations)) AND isnotnull(distributionCenters#69622.internalRefUUID))
(443) Project
Output [2]: [distributionCenters#69622.internalRefUUID AS _extract_internalRefUUID#70995, distributionCenters#69622.storageLocations AS _extract_storageLocations#70996]
Input [1]: [distributionCenters#69622]
(444) Generate
Input [2]: [_extract_internalRefUUID#70995, _extract_storageLocations#70996]
Arguments: explode(_extract_storageLocations#70996), [_extract_internalRefUUID#70995], false, [storageLocations#69623]
(445) Project
Output [3]: [_extract_internalRefUUID#70995 AS plantUuid#69625, storageLocations#69623.id AS storageLocId#69626, storageLocations#69623.internalRefUUID AS storageLocUuid#69627]
Input [2]: [_extract_internalRefUUID#70995, storageLocations#69623]
(446) BroadcastExchange
Input [3]: [plantUuid#69625, storageLocId#69626, storageLocUuid#69627]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32205]
(447) Scan parquet
Output [2]: [product#69628, plant#69629]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(448) Filter
Input [2]: [product#69628, plant#69629]
Condition : isnotnull(plant#69629)
(449) Exchange
Input [2]: [product#69628, plant#69629]
Arguments: hashpartitioning(product#69628, plant#69629, 25), REPARTITION_BY_NUM, [plan_id=31721]
(450) Scan parquet
Output [4]: [plant#69634, product#69635, productionAspect#69637, systemId#69632]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(451) Project
Output [4]: [plant#69634.internalRefUUID AS _extract_internalRefUUID#71382, product#69635.internalRefUUID AS _extract_internalRefUUID#71381, productionAspect#69637.productMovementPlants.unitOfIssue.code AS _extract_code#71379, productionAspect#69637.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71380]
Input [4]: [plant#69634, product#69635, productionAspect#69637, systemId#69632]
(452) Exchange
Input [4]: [_extract_internalRefUUID#71382, _extract_internalRefUUID#71381, _extract_code#71379, _extract_internalRefUUID#71380]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30875]
(453) Project
Output [4]: [_extract_code#71379, _extract_internalRefUUID#71380, _extract_internalRefUUID#71381, _extract_internalRefUUID#71382]
Input [4]: [_extract_internalRefUUID#71382, _extract_internalRefUUID#71381, _extract_code#71379, _extract_internalRefUUID#71380]
(454) BroadcastExchange
Input [4]: [_extract_code#71379, _extract_internalRefUUID#71380, _extract_internalRefUUID#71381, _extract_internalRefUUID#71382]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32165]
(455) BroadcastHashJoin
Left keys [4]: [coalesce(product#69628, ), isnull(product#69628), coalesce(plant#69629, ), isnull(plant#69629)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71381, ), isnull(_extract_internalRefUUID#71381), coalesce(_extract_internalRefUUID#71382, ), isnull(_extract_internalRefUUID#71382)]
Join type: LeftOuter
Join condition: None
(456) Project
Output [4]: [product#69628, plant#69629, _extract_code#71379, _extract_internalRefUUID#71380]
Input [6]: [product#69628, plant#69629, _extract_code#71379, _extract_internalRefUUID#71380, _extract_internalRefUUID#71381, _extract_internalRefUUID#71382]
(457) Scan parquet
Output [2]: [internalUUID#69642, systemId#69641]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(458) Project
Output [1]: [internalUUID#69642]
Input [2]: [internalUUID#69642, systemId#69641]
(459) Exchange
Input [1]: [internalUUID#69642]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30881]
(460) BroadcastExchange
Input [1]: [internalUUID#69642]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32173]
(461) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69629, ), isnull(plant#69629)]
Right keys [2]: [coalesce(internalUUID#69642, ), isnull(internalUUID#69642)]
Join type: LeftOuter
Join condition: None
(462) Project
Output [4]: [product#69628, plant#69629, _extract_code#71379, _extract_internalRefUUID#71380]
Input [5]: [product#69628, plant#69629, _extract_code#71379, _extract_internalRefUUID#71380, internalUUID#69642]
(463) Scan parquet
Output [4]: [internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662, systemId#69655]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(464) Filter
Input [4]: [internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662, systemId#69655]
Condition : isnotnull(internalUUID#69656)
(465) Project
Output [3]: [internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662]
Input [4]: [internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662, systemId#69655]
(466) Exchange
Input [3]: [internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30887]
(467) BroadcastExchange
Input [3]: [internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32183]
(468) BroadcastHashJoin
Left keys [1]: [product#69628]
Right keys [1]: [internalUUID#69656]
Join type: LeftOuter
Join condition: None
(469) Project
Output [2]: [plant#69629, filter(unitOfMeasureConversions#69662, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71379) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71380)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69660.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69660.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69667]
Input [7]: [product#69628, plant#69629, _extract_code#71379, _extract_internalRefUUID#71380, internalUUID#69656, baseUnitOfMeasure#69660, unitOfMeasureConversions#69662]
(470) Generate
Input [2]: [plant#69629, filteredUnitOfMeasureConversions#69667]
Arguments: explode(filteredUnitOfMeasureConversions#69667.quantityNumerator), [plant#69629, filteredUnitOfMeasureConversions#69667], true, [quantityNumerator#69668]
(471) Project
Output [2]: [plant#69629, filteredUnitOfMeasureConversions#69667.quantityDenominator AS _extract_quantityDenominator#70999]
Input [3]: [plant#69629, filteredUnitOfMeasureConversions#69667, quantityNumerator#69668]
(472) Generate
Input [2]: [plant#69629, _extract_quantityDenominator#70999]
Arguments: explode(_extract_quantityDenominator#70999), [plant#69629], true, [quantityDenominator#69669]
(473) Project
Output [1]: [plant#69629]
Input [2]: [plant#69629, quantityDenominator#69669]
(474) HashAggregate
Input [1]: [plant#69629]
Keys [1]: [plant#69629]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69629]
(475) Exchange
Input [1]: [plant#69629]
Arguments: hashpartitioning(plant#69629, 25), ENSURE_REQUIREMENTS, [plan_id=32199]
(476) HashAggregate
Input [1]: [plant#69629]
Keys [1]: [plant#69629]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69629]
(477) BroadcastHashJoin
Left keys [1]: [plantUuid#69625]
Right keys [1]: [plant#69629]
Join type: Inner
Join condition: None
(478) Project
Output [3]: [plantUuid#69625, storageLocId#69626, storageLocUuid#69627]
Input [4]: [plantUuid#69625, storageLocId#69626, storageLocUuid#69627, plant#69629]
(479) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69595.internalRefUUID, ), isnull(plant#69595.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#69625, ), isnull(plantUuid#69625)]
Join type: Inner
Join condition: None
(480) Generate
Input [9]: [systemId#69592, internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, stockQuantities#69597, plantUuid#69625, storageLocId#69626, storageLocUuid#69627]
Arguments: explode(filter(stockQuantities#69597, lambdafunction(((lambda x#69270.storageLocation.internalRefUuid = storageLocUuid#69627) AND (lambda x#69270.storageLocation.Id = storageLocId#69626)), lambda x#69270, false))), [systemId#69592, internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, plantUuid#69625], false, [stockQuantity#69672]
(481) ObjectHashAggregate
Input [7]: [systemId#69592, internalUUID#69593, product#69594, plant#69595, calculationDateTime#69596, plantUuid#69625, stockQuantity#69672]
Keys [5]: [plantUuid#69625, internalUUID#69593, plant#69595, product#69594, systemId#69592]
Functions [2]: [partial_first(calculationDateTime#69596, false), partial_collect_list(stockQuantity#69672, 0, 0)]
Aggregate Attributes [3]: [first#73918, valueSet#73919, buf#73920]
Results [8]: [plantUuid#69625, internalUUID#69593, plant#69595, product#69594, systemId#69592, first#73921, valueSet#73922, buf#73923]
(482) ObjectHashAggregate
Input [8]: [plantUuid#69625, internalUUID#69593, plant#69595, product#69594, systemId#69592, first#73921, valueSet#73922, buf#73923]
Keys [5]: [plantUuid#69625, internalUUID#69593, plant#69595, product#69594, systemId#69592]
Functions [2]: [first(calculationDateTime#69596, false), collect_list(stockQuantity#69672, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#69596)()#69321, collect_list(stockQuantity#69672, 0, 0)#69322]
Results [4]: [product#69594, plant#69595, first(calculationDateTime#69596)()#69321 AS calculationDateTime#69673, collect_list(stockQuantity#69672, 0, 0)#69322 AS stockQuantities#69674]
(483) Filter
Input [4]: [product#69594, plant#69595, calculationDateTime#69673, stockQuantities#69674]
Condition : ((size(stockQuantities#69674, true) > 0) AND isnotnull(calculationDateTime#69673))
(484) Generate
Input [4]: [product#69594, plant#69595, calculationDateTime#69673, stockQuantities#69674]
Arguments: explode(stockQuantities#69674), [product#69594, plant#69595, calculationDateTime#69673], false, [stockQuantity#69675]
(485) Filter
Input [4]: [product#69594, plant#69595, calculationDateTime#69673, stockQuantity#69675]
Condition : (((isnotnull(stockQuantity#69675.specialStockIndicator.code) AND isnotnull(stockQuantity#69675.stockType.code)) AND (stockQuantity#69675.specialStockIndicator.code = )) AND (stockQuantity#69675.stockType.code = 01))
(486) Project
Output [4]: [plant#69595.internalRefUUID AS plantUuid#69677, product#69594.internalRefUUID AS productUuid#69679, calculationDateTime#69673, stockQuantity#69675.storagelocation.id AS storageLocId#69680]
Input [4]: [product#69594, plant#69595, calculationDateTime#69673, stockQuantity#69675]
(487) Exchange
Input [4]: [plantUuid#69677, productUuid#69679, calculationDateTime#69673, storageLocId#69680]
Arguments: hashpartitioning(productUuid#69679, plantUuid#69677, 25), ENSURE_REQUIREMENTS, [plan_id=32265]
(488) Sort
Input [4]: [plantUuid#69677, productUuid#69679, calculationDateTime#69673, storageLocId#69680]
Arguments: [productUuid#69679 ASC NULLS FIRST, plantUuid#69677 ASC NULLS FIRST], false, 0
(489) Scan parquet
Output [3]: [product#69685, plant#69686, orderDateTime#69687]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(490) Filter
Input [3]: [product#69685, plant#69686, orderDateTime#69687]
Condition : ((isnotnull(orderDateTime#69687) AND isnotnull(plant#69686)) AND isnotnull(product#69685))
(491) Exchange
Input [3]: [product#69685, plant#69686, orderDateTime#69687]
Arguments: hashpartitioning(product#69685, plant#69686, 25), REPARTITION_BY_NUM, [plan_id=31766]
(492) Scan parquet
Output [4]: [plant#69691, product#69692, productionAspect#69694, systemId#69689]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(493) Project
Output [4]: [plant#69691.internalRefUUID AS _extract_internalRefUUID#71383, product#69692.internalRefUUID AS _extract_internalRefUUID#71384, productionAspect#69694.productMovementPlants.unitOfIssue.code AS _extract_code#71385, productionAspect#69694.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71386]
Input [4]: [plant#69691, product#69692, productionAspect#69694, systemId#69689]
(494) Exchange
Input [4]: [_extract_internalRefUUID#71383, _extract_internalRefUUID#71384, _extract_code#71385, _extract_internalRefUUID#71386]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30912]
(495) Project
Output [4]: [_extract_code#71385, _extract_internalRefUUID#71386, _extract_internalRefUUID#71384, _extract_internalRefUUID#71383]
Input [4]: [_extract_internalRefUUID#71383, _extract_internalRefUUID#71384, _extract_code#71385, _extract_internalRefUUID#71386]
(496) BroadcastExchange
Input [4]: [_extract_code#71385, _extract_internalRefUUID#71386, _extract_internalRefUUID#71384, _extract_internalRefUUID#71383]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32230]
(497) BroadcastHashJoin
Left keys [4]: [coalesce(product#69685, ), isnull(product#69685), coalesce(plant#69686, ), isnull(plant#69686)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71384, ), isnull(_extract_internalRefUUID#71384), coalesce(_extract_internalRefUUID#71383, ), isnull(_extract_internalRefUUID#71383)]
Join type: LeftOuter
Join condition: None
(498) Project
Output [5]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386]
Input [7]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386, _extract_internalRefUUID#71384, _extract_internalRefUUID#71383]
(499) Scan parquet
Output [2]: [internalUUID#69699, systemId#69698]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(500) Project
Output [1]: [internalUUID#69699]
Input [2]: [internalUUID#69699, systemId#69698]
(501) Exchange
Input [1]: [internalUUID#69699]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30918]
(502) BroadcastExchange
Input [1]: [internalUUID#69699]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32238]
(503) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69686, ), isnull(plant#69686)]
Right keys [2]: [coalesce(internalUUID#69699, ), isnull(internalUUID#69699)]
Join type: LeftOuter
Join condition: None
(504) Project
Output [5]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386]
Input [6]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386, internalUUID#69699]
(505) Scan parquet
Output [4]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719, systemId#69712]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(506) Filter
Input [4]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719, systemId#69712]
Condition : isnotnull(internalUUID#69713)
(507) Project
Output [3]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
Input [4]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719, systemId#69712]
(508) Exchange
Input [3]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30924]
(509) BroadcastExchange
Input [3]: [internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32247]
(510) BroadcastHashJoin
Left keys [1]: [product#69685]
Right keys [1]: [internalUUID#69713]
Join type: LeftOuter
Join condition: None
(511) Project
Output [4]: [product#69685, plant#69686, orderDateTime#69687, filter(unitOfMeasureConversions#69719, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71385) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71386)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#69717.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69717.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#69724]
Input [8]: [product#69685, plant#69686, orderDateTime#69687, _extract_code#71385, _extract_internalRefUUID#71386, internalUUID#69713, baseUnitOfMeasure#69717, unitOfMeasureConversions#69719]
(512) Generate
Input [4]: [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724]
Arguments: explode(filteredUnitOfMeasureConversions#69724.quantityNumerator), [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724], true, [quantityNumerator#69725]
(513) Project
Output [4]: [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724.quantityDenominator AS _extract_quantityDenominator#71006]
Input [5]: [product#69685, plant#69686, orderDateTime#69687, filteredUnitOfMeasureConversions#69724, quantityNumerator#69725]
(514) Generate
Input [4]: [product#69685, plant#69686, orderDateTime#69687, _extract_quantityDenominator#71006]
Arguments: explode(_extract_quantityDenominator#71006), [product#69685, plant#69686, orderDateTime#69687], true, [quantityDenominator#69726]
(515) Project
Output [3]: [product#69685, plant#69686, orderDateTime#69687]
Input [4]: [product#69685, plant#69686, orderDateTime#69687, quantityDenominator#69726]
(516) Sort
Input [3]: [product#69685, plant#69686, orderDateTime#69687]
Arguments: [product#69685 ASC NULLS FIRST, plant#69686 ASC NULLS FIRST], false, 0
(517) SortMergeJoin
Left keys [2]: [productUuid#69679, plantUuid#69677]
Right keys [2]: [product#69685, plant#69686]
Join type: Inner
Join condition: (calculationDateTime#69673 <= orderDateTime#69687)
(518) Project
Output [4]: [plantUuid#69677 AS plant#69729, productUuid#69679 AS product#69730, storageLocId#69680, calculationDateTime#69673]
Input [7]: [plantUuid#69677, productUuid#69679, calculationDateTime#69673, storageLocId#69680, product#69685, plant#69686, orderDateTime#69687]
(519) HashAggregate
Input [4]: [plant#69729, product#69730, storageLocId#69680, calculationDateTime#69673]
Keys [3]: [plant#69729, product#69730, storageLocId#69680]
Functions [1]: [partial_max(calculationDateTime#69673)]
Aggregate Attributes [1]: [max#73916]
Results [4]: [plant#69729, product#69730, storageLocId#69680, max#73917]
(520) HashAggregate
Input [4]: [plant#69729, product#69730, storageLocId#69680, max#73917]
Keys [3]: [plant#69729, product#69730, storageLocId#69680]
Functions [1]: [max(calculationDateTime#69673)]
Aggregate Attributes [1]: [max(calculationDateTime#69673)#69456]
Results [4]: [plant#69729, product#69730, storageLocId#69680, max(calculationDateTime#69673)#69456 AS max_calc_datetime#69450]
(521) Exchange
Input [4]: [plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
Arguments: hashpartitioning(coalesce(plant#69729, ), isnull(plant#69729), coalesce(product#69730, ), isnull(product#69730), coalesce(storageLocId#69680, ), isnull(storageLocId#69680), coalesce(max_calc_datetime#69450, 1970-01-01 00:00:00), isnull(max_calc_datetime#69450), 25), ENSURE_REQUIREMENTS, [plan_id=32284]
(522) Sort
Input [4]: [plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
Arguments: [coalesce(plant#69729, ) ASC NULLS FIRST, isnull(plant#69729) ASC NULLS FIRST, coalesce(product#69730, ) ASC NULLS FIRST, isnull(product#69730) ASC NULLS FIRST, coalesce(storageLocId#69680, ) ASC NULLS FIRST, isnull(storageLocId#69680) ASC NULLS FIRST, coalesce(max_calc_datetime#69450, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(max_calc_datetime#69450) ASC NULLS FIRST], false, 0
(523) SortMergeJoin
Left keys [8]: [coalesce(plant#69448, ), isnull(plant#69448), coalesce(product#69449, ), isnull(product#69449), coalesce(storageLocId#69373, ), isnull(storageLocId#69373), coalesce(calculationDateTime#69308, 1970-01-01 00:00:00), isnull(calculationDateTime#69308)]
Right keys [8]: [coalesce(plant#69729, ), isnull(plant#69729), coalesce(product#69730, ), isnull(product#69730), coalesce(storageLocId#69680, ), isnull(storageLocId#69680), coalesce(max_calc_datetime#69450, 1970-01-01 00:00:00), isnull(max_calc_datetime#69450)]
Join type: Inner
Join condition: None
(524) Project
Output [3]: [plant#69448, product#69449, calculationDateTime#69308]
Input [8]: [plant#69448, product#69449, storageLocId#69373, calculationDateTime#69308, plant#69729, product#69730, storageLocId#69680, max_calc_datetime#69450]
(525) HashAggregate
Input [3]: [plant#69448, product#69449, calculationDateTime#69308]
Keys [2]: [product#69449, plant#69448]
Functions [1]: [partial_min(calculationDateTime#69308)]
Aggregate Attributes [1]: [min#73906]
Results [3]: [product#69449, plant#69448, min#73907]
(526) Exchange
Input [3]: [product#69449, plant#69448, min#73907]
Arguments: hashpartitioning(product#69449, plant#69448, 25), ENSURE_REQUIREMENTS, [plan_id=32298]
(527) HashAggregate
Input [3]: [product#69449, plant#69448, min#73907]
Keys [2]: [product#69449, plant#69448]
Functions [1]: [min(calculationDateTime#69308)]
Aggregate Attributes [1]: [min(calculationDateTime#69308)#69768]
Results [3]: [product#69449, plant#69448, min(calculationDateTime#69308)#69768 AS minCalculationDateTime#69754]
(528) Sort
Input [3]: [product#69449, plant#69448, minCalculationDateTime#69754]
Arguments: [product#69449 ASC NULLS FIRST, plant#69448 ASC NULLS FIRST], false, 0
(529) SortMergeJoin
Left keys [2]: [product#69015, plant#69016]
Right keys [2]: [product#69449, plant#69448]
Join type: LeftOuter
Join condition: None
(530) Project
Output [4]: [product#69015, plant#69016, timezonecode#69080, CASE WHEN isnotnull(minCalculationDateTime#69754) THEN minCalculationDateTime#69754 ELSE orderDateTime#69017 END AS stockDatetime#69867]
Input [7]: [product#69015, plant#69016, orderDateTime#69017, timezonecode#69080, product#69449, plant#69448, minCalculationDateTime#69754]
(531) Exchange
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: hashpartitioning(coalesce(plant#69016, ), isnull(plant#69016), coalesce(product#69015, ), isnull(product#69015), 25), ENSURE_REQUIREMENTS, [plan_id=32620]
(532) Sort
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: [coalesce(plant#69016, ) ASC NULLS FIRST, isnull(plant#69016) ASC NULLS FIRST, coalesce(product#69015, ) ASC NULLS FIRST, isnull(product#69015) ASC NULLS FIRST], false, 0
(533) Exchange
Input [3]: [product#70356, plant#70357, orderDateTime#70358]
Arguments: hashpartitioning(product#70356, plant#70357, 25), REPARTITION_BY_NUM, [plan_id=30944]
(534) Project
Output [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
Input [8]: [internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367, systemId#70360]
(535) Exchange
Input [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30947]
(536) Project
Output [1]: [struct(systemId, systemId#70360, internalUUID, internalUUID#70361, plant, plant#70362, product, product#70363, blockedForReplenishmentStartingFrom, blockedForReplenishmentStartingFrom#70364, productionAspect, productionAspect#70365, salesPlant, salesPlant#70366, listing, listing#70367) AS productPlant#70368]
Input [8]: [systemId#70360, internalUUID#70361, plant#70362, product#70363, blockedForReplenishmentStartingFrom#70364, productionAspect#70365, salesPlant#70366, listing#70367]
(537) BroadcastExchange
Input [1]: [productPlant#70368]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID), coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID)),false), [plan_id=32314]
(538) BroadcastHashJoin
Left keys [4]: [coalesce(product#70356, ), isnull(product#70356), coalesce(plant#70357, ), isnull(plant#70357)]
Right keys [4]: [coalesce(productPlant#70368.product.internalRefUUID, ), isnull(productPlant#70368.product.internalRefUUID), coalesce(productPlant#70368.plant.internalRefUUID, ), isnull(productPlant#70368.plant.internalRefUUID)]
Join type: LeftOuter
Join condition: None
(539) Scan parquet
Output [3]: [internalUUID#70370, address#70376, systemId#70369]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string,address:struct<timeZone:struct<code:string>>>
(540) Project
Output [2]: [internalUUID#70370, address#70376.timeZone.code AS timezonecode#70381]
Input [3]: [internalUUID#70370, address#70376, systemId#70369]
(541) Exchange
Input [2]: [internalUUID#70370, timezonecode#70381]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30952]
(542) BroadcastExchange
Input [2]: [internalUUID#70370, timezonecode#70381]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32332]
(543) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70357, ), isnull(plant#70357)]
Right keys [2]: [coalesce(internalUUID#70370, ), isnull(internalUUID#70370)]
Join type: LeftOuter
Join condition: None
(544) Project
Output [5]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, CASE WHEN isnotnull(timezonecode#70381) THEN timezonecode#70381 ELSE UTC END AS timezonecode#70382]
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, internalUUID#70370, timezonecode#70381]
(545) Scan parquet
Output [4]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390, systemId#70383]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(546) Filter
Input [4]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390, systemId#70383]
Condition : isnotnull(internalUUID#70384)
(547) Project
Output [3]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
Input [4]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390, systemId#70383]
(548) Exchange
Input [3]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30960]
(549) BroadcastExchange
Input [3]: [internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32339]
(550) BroadcastHashJoin
Left keys [1]: [product#70356]
Right keys [1]: [internalUUID#70384]
Join type: LeftOuter
Join condition: None
(551) Project
Output [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, filter(unitOfMeasureConversions#70390, lambdafunction((((lambda u#69110.measurementUnit1.code = productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.code) AND (lambda u#69110.measurementUnit1.internalRefUUID = productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70388.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70388.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70396]
Input [8]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, internalUUID#70384, baseUnitOfMeasure#70388, unitOfMeasureConversions#70390]
(552) Project
Output [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, filteredUnitOfMeasureConversions#70396.quantityDenominator AS _extract_quantityDenominator#71131, filteredUnitOfMeasureConversions#70396.quantityNumerator AS _extract_quantityNumerator#71132]
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, filteredUnitOfMeasureConversions#70396]
(553) Generate
Input [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, _extract_quantityDenominator#71131, _extract_quantityNumerator#71132]
Arguments: explode(_extract_quantityNumerator#71132), [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, _extract_quantityDenominator#71131], true, [quantityNumerator#70398]
(554) Generate
Input [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, _extract_quantityDenominator#71131, quantityNumerator#70398]
Arguments: explode(_extract_quantityDenominator#71131), [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, quantityNumerator#70398], true, [quantityDenominator#70399]
(555) Project
Output [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, (cast(quantityNumerator#70398 as double) / cast(quantityDenominator#70399 as double)) AS outboundUnit#70400]
Input [7]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, quantityNumerator#70398, quantityDenominator#70399]
(556) Project
Output [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, CASE WHEN (isnull(outboundUnit#70400) OR (outboundUnit#70400 <= 0.0)) THEN 1.0 ELSE outboundUnit#70400 END AS outboundUnit#70401]
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70400]
(557) Sort
Input [6]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401]
Arguments: [product#70356 ASC NULLS FIRST, plant#70357 ASC NULLS FIRST], false, 0
(558) Scan parquet
Output [6]: [internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407, systemId#70402]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(559) Filter
Input [6]: [internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407, systemId#70402]
Condition : (isnotnull(plant#70405.internalRefUUID) AND isnotnull(product#70404.internalRefUUID))
(560) Project
Output [6]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407]
Input [6]: [internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407, systemId#70402]
(561) Exchange
Input [6]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30994]
(562) BroadcastExchange
Input [6]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32399]
(563) Scan parquet
Output [2]: [marketUnit#70408, distributionCenters#70414]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(564) Filter
Input [2]: [marketUnit#70408, distributionCenters#70414]
Condition : (((marketUnit#70408 <=> Daily_grocery) AND (size(distributionCenters#70414, true) > 0)) AND isnotnull(distributionCenters#70414))
(565) Project
Output [1]: [distributionCenters#70414]
Input [2]: [marketUnit#70408, distributionCenters#70414]
(566) Exchange
Input [1]: [distributionCenters#70414]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31002]
(567) Generate
Input [1]: [distributionCenters#70414]
Arguments: explode(distributionCenters#70414), false, [distributionCenters#70430]
(568) Filter
Input [1]: [distributionCenters#70430]
Condition : (((size(distributionCenters#70430.storageLocations, true) > 0) AND isnotnull(distributionCenters#70430.storageLocations)) AND isnotnull(distributionCenters#70430.internalRefUUID))
(569) Project
Output [2]: [distributionCenters#70430.internalRefUUID AS _extract_internalRefUUID#71019, distributionCenters#70430.storageLocations AS _extract_storageLocations#71020]
Input [1]: [distributionCenters#70430]
(570) Generate
Input [2]: [_extract_internalRefUUID#71019, _extract_storageLocations#71020]
Arguments: explode(_extract_storageLocations#71020), [_extract_internalRefUUID#71019], false, [storageLocations#70431]
(571) Project
Output [3]: [_extract_internalRefUUID#71019 AS plantUuid#70433, storageLocations#70431.id AS storageLocId#70434, storageLocations#70431.internalRefUUID AS storageLocUuid#70435]
Input [2]: [_extract_internalRefUUID#71019, storageLocations#70431]
(572) BroadcastExchange
Input [3]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32393]
(573) Scan parquet
Output [2]: [product#70437, plant#70438]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(574) Filter
Input [2]: [product#70437, plant#70438]
Condition : isnotnull(plant#70438)
(575) Exchange
Input [2]: [product#70437, plant#70438]
Arguments: hashpartitioning(product#70437, plant#70438, 25), REPARTITION_BY_NUM, [plan_id=31808]
(576) Scan parquet
Output [4]: [plant#70444, product#70445, productionAspect#70447, systemId#70442]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(577) Project
Output [4]: [plant#70444.internalRefUUID AS _extract_internalRefUUID#71396, product#70445.internalRefUUID AS _extract_internalRefUUID#71395, productionAspect#70447.productMovementPlants.unitOfIssue.code AS _extract_code#71397, productionAspect#70447.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71398]
Input [4]: [plant#70444, product#70445, productionAspect#70447, systemId#70442]
(578) Exchange
Input [4]: [_extract_internalRefUUID#71396, _extract_internalRefUUID#71395, _extract_code#71397, _extract_internalRefUUID#71398]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31023]
(579) Project
Output [4]: [_extract_code#71397, _extract_internalRefUUID#71398, _extract_internalRefUUID#71395, _extract_internalRefUUID#71396]
Input [4]: [_extract_internalRefUUID#71396, _extract_internalRefUUID#71395, _extract_code#71397, _extract_internalRefUUID#71398]
(580) BroadcastExchange
Input [4]: [_extract_code#71397, _extract_internalRefUUID#71398, _extract_internalRefUUID#71395, _extract_internalRefUUID#71396]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32361]
(581) BroadcastHashJoin
Left keys [4]: [coalesce(product#70437, ), isnull(product#70437), coalesce(plant#70438, ), isnull(plant#70438)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71395, ), isnull(_extract_internalRefUUID#71395), coalesce(_extract_internalRefUUID#71396, ), isnull(_extract_internalRefUUID#71396)]
Join type: LeftOuter
Join condition: None
(582) Project
Output [4]: [product#70437, plant#70438, _extract_code#71397, _extract_internalRefUUID#71398]
Input [6]: [product#70437, plant#70438, _extract_code#71397, _extract_internalRefUUID#71398, _extract_internalRefUUID#71395, _extract_internalRefUUID#71396]
(583) Scan parquet
Output [2]: [internalUUID#70454, systemId#70453]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(584) Project
Output [1]: [internalUUID#70454]
Input [2]: [internalUUID#70454, systemId#70453]
(585) Exchange
Input [1]: [internalUUID#70454]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31033]
(586) BroadcastExchange
Input [1]: [internalUUID#70454]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32367]
(587) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70438, ), isnull(plant#70438)]
Right keys [2]: [coalesce(internalUUID#70454, ), isnull(internalUUID#70454)]
Join type: LeftOuter
Join condition: None
(588) Project
Output [4]: [product#70437, plant#70438, _extract_code#71397, _extract_internalRefUUID#71398]
Input [5]: [product#70437, plant#70438, _extract_code#71397, _extract_internalRefUUID#71398, internalUUID#70454]
(589) Scan parquet
Output [4]: [internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478, systemId#70471]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(590) Filter
Input [4]: [internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478, systemId#70471]
Condition : isnotnull(internalUUID#70472)
(591) Project
Output [3]: [internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478]
Input [4]: [internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478, systemId#70471]
(592) Exchange
Input [3]: [internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31046]
(593) BroadcastExchange
Input [3]: [internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32375]
(594) BroadcastHashJoin
Left keys [1]: [product#70437]
Right keys [1]: [internalUUID#70472]
Join type: LeftOuter
Join condition: None
(595) Project
Output [2]: [plant#70438, filter(unitOfMeasureConversions#70478, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71397) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71398)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70476.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70476.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70483]
Input [7]: [product#70437, plant#70438, _extract_code#71397, _extract_internalRefUUID#71398, internalUUID#70472, baseUnitOfMeasure#70476, unitOfMeasureConversions#70478]
(596) Generate
Input [2]: [plant#70438, filteredUnitOfMeasureConversions#70483]
Arguments: explode(filteredUnitOfMeasureConversions#70483.quantityNumerator), [plant#70438, filteredUnitOfMeasureConversions#70483], true, [quantityNumerator#70484]
(597) Project
Output [2]: [plant#70438, filteredUnitOfMeasureConversions#70483.quantityDenominator AS _extract_quantityDenominator#71023]
Input [3]: [plant#70438, filteredUnitOfMeasureConversions#70483, quantityNumerator#70484]
(598) Generate
Input [2]: [plant#70438, _extract_quantityDenominator#71023]
Arguments: explode(_extract_quantityDenominator#71023), [plant#70438], true, [quantityDenominator#70487]
(599) Project
Output [1]: [plant#70438]
Input [2]: [plant#70438, quantityDenominator#70487]
(600) HashAggregate
Input [1]: [plant#70438]
Keys [1]: [plant#70438]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70438]
(601) Exchange
Input [1]: [plant#70438]
Arguments: hashpartitioning(plant#70438, 25), ENSURE_REQUIREMENTS, [plan_id=32389]
(602) HashAggregate
Input [1]: [plant#70438]
Keys [1]: [plant#70438]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70438]
(603) BroadcastHashJoin
Left keys [1]: [plantUuid#70433]
Right keys [1]: [plant#70438]
Join type: Inner
Join condition: None
(604) Project
Output [3]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
Input [4]: [plantUuid#70433, storageLocId#70434, storageLocUuid#70435, plant#70438]
(605) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70405.internalRefUUID, ), isnull(plant#70405.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#70433, ), isnull(plantUuid#70433)]
Join type: Inner
Join condition: None
(606) Generate
Input [9]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, stockQuantities#70407, plantUuid#70433, storageLocId#70434, storageLocUuid#70435]
Arguments: explode(filter(stockQuantities#70407, lambdafunction(((lambda x#69270.storageLocation.internalRefUuid = storageLocUuid#70435) AND (lambda x#69270.storageLocation.Id = storageLocId#70434)), lambda x#69270, false))), [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, plantUuid#70433], false, [stockQuantity#70498]
(607) ObjectHashAggregate
Input [7]: [systemId#70402, internalUUID#70403, product#70404, plant#70405, calculationDateTime#70406, plantUuid#70433, stockQuantity#70498]
Keys [5]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402]
Functions [2]: [partial_first(calculationDateTime#70406, false), partial_collect_list(stockQuantity#70498, 0, 0)]
Aggregate Attributes [3]: [first#73946, valueSet#73947, buf#73948]
Results [8]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402, first#73949, valueSet#73950, buf#73951]
(608) ObjectHashAggregate
Input [8]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402, first#73949, valueSet#73950, buf#73951]
Keys [5]: [plantUuid#70433, internalUUID#70403, plant#70405, product#70404, systemId#70402]
Functions [2]: [first(calculationDateTime#70406, false), collect_list(stockQuantity#70498, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#70406)()#69321, collect_list(stockQuantity#70498, 0, 0)#69322]
Results [4]: [product#70404, plant#70405, first(calculationDateTime#70406)()#69321 AS calculationDateTime#70499, collect_list(stockQuantity#70498, 0, 0)#69322 AS stockQuantities#70500]
(609) Filter
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantities#70500]
Condition : ((size(stockQuantities#70500, true) > 0) AND isnotnull(calculationDateTime#70499))
(610) Generate
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantities#70500]
Arguments: explode(stockQuantities#70500), [product#70404, plant#70405, calculationDateTime#70499], false, [stockQuantity#70501]
(611) Filter
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantity#70501]
Condition : (((isnotnull(stockQuantity#70501.specialStockIndicator.code) AND isnotnull(stockQuantity#70501.stockType.code)) AND (stockQuantity#70501.specialStockIndicator.code = )) AND (stockQuantity#70501.stockType.code = 01))
(612) Project
Output [5]: [plant#70405.internalRefUUID AS plantUuid#70503, product#70404.internalRefUUID AS productUuid#70505, calculationDateTime#70499, stockQuantity#70501.storagelocation.id AS storageLocId#70506, stockQuantity#70501.quantity.measure AS quantity#70508]
Input [4]: [product#70404, plant#70405, calculationDateTime#70499, stockQuantity#70501]
(613) Exchange
Input [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: hashpartitioning(productUuid#70505, plantUuid#70503, 25), ENSURE_REQUIREMENTS, [plan_id=32445]
(614) Sort
Input [5]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508]
Arguments: [productUuid#70505 ASC NULLS FIRST, plantUuid#70503 ASC NULLS FIRST], false, 0
(615) Scan parquet
Output [3]: [product#70513, plant#70514, orderDateTime#70515]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(616) Filter
Input [3]: [product#70513, plant#70514, orderDateTime#70515]
Condition : ((isnotnull(orderDateTime#70515) AND isnotnull(plant#70514)) AND isnotnull(product#70513))
(617) Exchange
Input [3]: [product#70513, plant#70514, orderDateTime#70515]
Arguments: hashpartitioning(product#70513, plant#70514, 25), REPARTITION_BY_NUM, [plan_id=31854]
(618) Scan parquet
Output [4]: [plant#70519, product#70520, productionAspect#70522, systemId#70517]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(619) Project
Output [4]: [plant#70519.internalRefUUID AS _extract_internalRefUUID#71401, product#70520.internalRefUUID AS _extract_internalRefUUID#71402, productionAspect#70522.productMovementPlants.unitOfIssue.code AS _extract_code#71399, productionAspect#70522.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71400]
Input [4]: [plant#70519, product#70520, productionAspect#70522, systemId#70517]
(620) Exchange
Input [4]: [_extract_internalRefUUID#71401, _extract_internalRefUUID#71402, _extract_code#71399, _extract_internalRefUUID#71400]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31097]
(621) Project
Output [4]: [_extract_code#71399, _extract_internalRefUUID#71400, _extract_internalRefUUID#71402, _extract_internalRefUUID#71401]
Input [4]: [_extract_internalRefUUID#71401, _extract_internalRefUUID#71402, _extract_code#71399, _extract_internalRefUUID#71400]
(622) BroadcastExchange
Input [4]: [_extract_code#71399, _extract_internalRefUUID#71400, _extract_internalRefUUID#71402, _extract_internalRefUUID#71401]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32423]
(623) BroadcastHashJoin
Left keys [4]: [coalesce(product#70513, ), isnull(product#70513), coalesce(plant#70514, ), isnull(plant#70514)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71402, ), isnull(_extract_internalRefUUID#71402), coalesce(_extract_internalRefUUID#71401, ), isnull(_extract_internalRefUUID#71401)]
Join type: LeftOuter
Join condition: None
(624) Project
Output [5]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400]
Input [7]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400, _extract_internalRefUUID#71402, _extract_internalRefUUID#71401]
(625) Scan parquet
Output [2]: [internalUUID#70529, systemId#70528]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(626) Project
Output [1]: [internalUUID#70529]
Input [2]: [internalUUID#70529, systemId#70528]
(627) Exchange
Input [1]: [internalUUID#70529]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31105]
(628) BroadcastExchange
Input [1]: [internalUUID#70529]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32429]
(629) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70514, ), isnull(plant#70514)]
Right keys [2]: [coalesce(internalUUID#70529, ), isnull(internalUUID#70529)]
Join type: LeftOuter
Join condition: None
(630) Project
Output [5]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400]
Input [6]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400, internalUUID#70529]
(631) Scan parquet
Output [4]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549, systemId#70542]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(632) Filter
Input [4]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549, systemId#70542]
Condition : isnotnull(internalUUID#70543)
(633) Project
Output [3]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
Input [4]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549, systemId#70542]
(634) Exchange
Input [3]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31115]
(635) BroadcastExchange
Input [3]: [internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32434]
(636) BroadcastHashJoin
Left keys [1]: [product#70513]
Right keys [1]: [internalUUID#70543]
Join type: LeftOuter
Join condition: None
(637) Project
Output [4]: [product#70513, plant#70514, orderDateTime#70515, filter(unitOfMeasureConversions#70549, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71399) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71400)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70547.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70547.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70554]
Input [8]: [product#70513, plant#70514, orderDateTime#70515, _extract_code#71399, _extract_internalRefUUID#71400, internalUUID#70543, baseUnitOfMeasure#70547, unitOfMeasureConversions#70549]
(638) Generate
Input [4]: [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554]
Arguments: explode(filteredUnitOfMeasureConversions#70554.quantityNumerator), [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554], true, [quantityNumerator#70555]
(639) Project
Output [4]: [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554.quantityDenominator AS _extract_quantityDenominator#71032]
Input [5]: [product#70513, plant#70514, orderDateTime#70515, filteredUnitOfMeasureConversions#70554, quantityNumerator#70555]
(640) Generate
Input [4]: [product#70513, plant#70514, orderDateTime#70515, _extract_quantityDenominator#71032]
Arguments: explode(_extract_quantityDenominator#71032), [product#70513, plant#70514, orderDateTime#70515], true, [quantityDenominator#70556]
(641) Project
Output [3]: [product#70513, plant#70514, orderDateTime#70515]
Input [4]: [product#70513, plant#70514, orderDateTime#70515, quantityDenominator#70556]
(642) Sort
Input [3]: [product#70513, plant#70514, orderDateTime#70515]
Arguments: [product#70513 ASC NULLS FIRST, plant#70514 ASC NULLS FIRST], false, 0
(643) SortMergeJoin
Left keys [2]: [productUuid#70505, plantUuid#70503]
Right keys [2]: [product#70513, plant#70514]
Join type: Inner
Join condition: (calculationDateTime#70499 <= orderDateTime#70515)
(644) Project
Output [5]: [plantUuid#70503 AS plant#70559, productUuid#70505 AS product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Input [8]: [plantUuid#70503, productUuid#70505, calculationDateTime#70499, storageLocId#70506, quantity#70508, product#70513, plant#70514, orderDateTime#70515]
(645) Exchange
Input [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: hashpartitioning(coalesce(plant#70559, ), isnull(plant#70559), coalesce(product#70560, ), isnull(product#70560), coalesce(storageLocId#70506, ), isnull(storageLocId#70506), coalesce(calculationDateTime#70499, 1970-01-01 00:00:00), isnull(calculationDateTime#70499), 25), ENSURE_REQUIREMENTS, [plan_id=32569]
(646) Sort
Input [5]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508]
Arguments: [coalesce(plant#70559, ) ASC NULLS FIRST, isnull(plant#70559) ASC NULLS FIRST, coalesce(product#70560, ) ASC NULLS FIRST, isnull(product#70560) ASC NULLS FIRST, coalesce(storageLocId#70506, ) ASC NULLS FIRST, isnull(storageLocId#70506) ASC NULLS FIRST, coalesce(calculationDateTime#70499, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(calculationDateTime#70499) ASC NULLS FIRST], false, 0
(647) Scan parquet
Output [6]: [internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566, systemId#70561]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(648) Filter
Input [6]: [internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566, systemId#70561]
Condition : (isnotnull(plant#70564.internalRefUUID) AND isnotnull(product#70563.internalRefUUID))
(649) Project
Output [6]: [systemId#70561, internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566]
Input [6]: [internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566, systemId#70561]
(650) Exchange
Input [6]: [systemId#70561, internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31148]
(651) BroadcastExchange
Input [6]: [systemId#70561, internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32502]
(652) Scan parquet
Output [2]: [marketUnit#70567, distributionCenters#70573]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(653) Filter
Input [2]: [marketUnit#70567, distributionCenters#70573]
Condition : (((marketUnit#70567 <=> Daily_grocery) AND (size(distributionCenters#70573, true) > 0)) AND isnotnull(distributionCenters#70573))
(654) Project
Output [1]: [distributionCenters#70573]
Input [2]: [marketUnit#70567, distributionCenters#70573]
(655) Exchange
Input [1]: [distributionCenters#70573]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31152]
(656) Generate
Input [1]: [distributionCenters#70573]
Arguments: explode(distributionCenters#70573), false, [distributionCenters#70589]
(657) Filter
Input [1]: [distributionCenters#70589]
Condition : (((size(distributionCenters#70589.storageLocations, true) > 0) AND isnotnull(distributionCenters#70589.storageLocations)) AND isnotnull(distributionCenters#70589.internalRefUUID))
(658) Project
Output [2]: [distributionCenters#70589.internalRefUUID AS _extract_internalRefUUID#71039, distributionCenters#70589.storageLocations AS _extract_storageLocations#71040]
Input [1]: [distributionCenters#70589]
(659) Generate
Input [2]: [_extract_internalRefUUID#71039, _extract_storageLocations#71040]
Arguments: explode(_extract_storageLocations#71040), [_extract_internalRefUUID#71039], false, [storageLocations#70590]
(660) Project
Output [3]: [_extract_internalRefUUID#71039 AS plantUuid#70592, storageLocations#70590.id AS storageLocId#70593, storageLocations#70590.internalRefUUID AS storageLocUuid#70594]
Input [2]: [_extract_internalRefUUID#71039, storageLocations#70590]
(661) BroadcastExchange
Input [3]: [plantUuid#70592, storageLocId#70593, storageLocUuid#70594]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32495]
(662) Scan parquet
Output [2]: [product#70595, plant#70596]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(663) Filter
Input [2]: [product#70595, plant#70596]
Condition : isnotnull(plant#70596)
(664) Exchange
Input [2]: [product#70595, plant#70596]
Arguments: hashpartitioning(product#70595, plant#70596, 25), REPARTITION_BY_NUM, [plan_id=31880]
(665) Scan parquet
Output [4]: [plant#70601, product#70602, productionAspect#70604, systemId#70599]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(666) Project
Output [4]: [plant#70601.internalRefUUID AS _extract_internalRefUUID#71412, product#70602.internalRefUUID AS _extract_internalRefUUID#71411, productionAspect#70604.productMovementPlants.unitOfIssue.code AS _extract_code#71413, productionAspect#70604.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71414]
Input [4]: [plant#70601, product#70602, productionAspect#70604, systemId#70599]
(667) Exchange
Input [4]: [_extract_internalRefUUID#71412, _extract_internalRefUUID#71411, _extract_code#71413, _extract_internalRefUUID#71414]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31177]
(668) Project
Output [4]: [_extract_code#71413, _extract_internalRefUUID#71414, _extract_internalRefUUID#71411, _extract_internalRefUUID#71412]
Input [4]: [_extract_internalRefUUID#71412, _extract_internalRefUUID#71411, _extract_code#71413, _extract_internalRefUUID#71414]
(669) BroadcastExchange
Input [4]: [_extract_code#71413, _extract_internalRefUUID#71414, _extract_internalRefUUID#71411, _extract_internalRefUUID#71412]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32457]
(670) BroadcastHashJoin
Left keys [4]: [coalesce(product#70595, ), isnull(product#70595), coalesce(plant#70596, ), isnull(plant#70596)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71411, ), isnull(_extract_internalRefUUID#71411), coalesce(_extract_internalRefUUID#71412, ), isnull(_extract_internalRefUUID#71412)]
Join type: LeftOuter
Join condition: None
(671) Project
Output [4]: [product#70595, plant#70596, _extract_code#71413, _extract_internalRefUUID#71414]
Input [6]: [product#70595, plant#70596, _extract_code#71413, _extract_internalRefUUID#71414, _extract_internalRefUUID#71411, _extract_internalRefUUID#71412]
(672) Scan parquet
Output [2]: [internalUUID#70609, systemId#70608]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(673) Project
Output [1]: [internalUUID#70609]
Input [2]: [internalUUID#70609, systemId#70608]
(674) Exchange
Input [1]: [internalUUID#70609]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31184]
(675) BroadcastExchange
Input [1]: [internalUUID#70609]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32465]
(676) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70596, ), isnull(plant#70596)]
Right keys [2]: [coalesce(internalUUID#70609, ), isnull(internalUUID#70609)]
Join type: LeftOuter
Join condition: None
(677) Project
Output [4]: [product#70595, plant#70596, _extract_code#71413, _extract_internalRefUUID#71414]
Input [5]: [product#70595, plant#70596, _extract_code#71413, _extract_internalRefUUID#71414, internalUUID#70609]
(678) Scan parquet
Output [4]: [internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629, systemId#70622]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(679) Filter
Input [4]: [internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629, systemId#70622]
Condition : isnotnull(internalUUID#70623)
(680) Project
Output [3]: [internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629]
Input [4]: [internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629, systemId#70622]
(681) Exchange
Input [3]: [internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31195]
(682) BroadcastExchange
Input [3]: [internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32474]
(683) BroadcastHashJoin
Left keys [1]: [product#70595]
Right keys [1]: [internalUUID#70623]
Join type: LeftOuter
Join condition: None
(684) Project
Output [2]: [plant#70596, filter(unitOfMeasureConversions#70629, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71413) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71414)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70627.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70627.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70634]
Input [7]: [product#70595, plant#70596, _extract_code#71413, _extract_internalRefUUID#71414, internalUUID#70623, baseUnitOfMeasure#70627, unitOfMeasureConversions#70629]
(685) Generate
Input [2]: [plant#70596, filteredUnitOfMeasureConversions#70634]
Arguments: explode(filteredUnitOfMeasureConversions#70634.quantityNumerator), [plant#70596, filteredUnitOfMeasureConversions#70634], true, [quantityNumerator#70635]
(686) Project
Output [2]: [plant#70596, filteredUnitOfMeasureConversions#70634.quantityDenominator AS _extract_quantityDenominator#71043]
Input [3]: [plant#70596, filteredUnitOfMeasureConversions#70634, quantityNumerator#70635]
(687) Generate
Input [2]: [plant#70596, _extract_quantityDenominator#71043]
Arguments: explode(_extract_quantityDenominator#71043), [plant#70596], true, [quantityDenominator#70636]
(688) Project
Output [1]: [plant#70596]
Input [2]: [plant#70596, quantityDenominator#70636]
(689) HashAggregate
Input [1]: [plant#70596]
Keys [1]: [plant#70596]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70596]
(690) Exchange
Input [1]: [plant#70596]
Arguments: hashpartitioning(plant#70596, 25), ENSURE_REQUIREMENTS, [plan_id=32491]
(691) HashAggregate
Input [1]: [plant#70596]
Keys [1]: [plant#70596]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70596]
(692) BroadcastHashJoin
Left keys [1]: [plantUuid#70592]
Right keys [1]: [plant#70596]
Join type: Inner
Join condition: None
(693) Project
Output [3]: [plantUuid#70592, storageLocId#70593, storageLocUuid#70594]
Input [4]: [plantUuid#70592, storageLocId#70593, storageLocUuid#70594, plant#70596]
(694) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70564.internalRefUUID, ), isnull(plant#70564.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#70592, ), isnull(plantUuid#70592)]
Join type: Inner
Join condition: None
(695) Generate
Input [9]: [systemId#70561, internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, stockQuantities#70566, plantUuid#70592, storageLocId#70593, storageLocUuid#70594]
Arguments: explode(filter(stockQuantities#70566, lambdafunction(((lambda x#69270.storageLocation.internalRefUuid = storageLocUuid#70594) AND (lambda x#69270.storageLocation.Id = storageLocId#70593)), lambda x#69270, false))), [systemId#70561, internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, plantUuid#70592], false, [stockQuantity#70639]
(696) ObjectHashAggregate
Input [7]: [systemId#70561, internalUUID#70562, product#70563, plant#70564, calculationDateTime#70565, plantUuid#70592, stockQuantity#70639]
Keys [5]: [plantUuid#70592, internalUUID#70562, plant#70564, product#70563, systemId#70561]
Functions [2]: [partial_first(calculationDateTime#70565, false), partial_collect_list(stockQuantity#70639, 0, 0)]
Aggregate Attributes [3]: [first#73968, valueSet#73969, buf#73970]
Results [8]: [plantUuid#70592, internalUUID#70562, plant#70564, product#70563, systemId#70561, first#73971, valueSet#73972, buf#73973]
(697) ObjectHashAggregate
Input [8]: [plantUuid#70592, internalUUID#70562, plant#70564, product#70563, systemId#70561, first#73971, valueSet#73972, buf#73973]
Keys [5]: [plantUuid#70592, internalUUID#70562, plant#70564, product#70563, systemId#70561]
Functions [2]: [first(calculationDateTime#70565, false), collect_list(stockQuantity#70639, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#70565)()#69321, collect_list(stockQuantity#70639, 0, 0)#69322]
Results [4]: [product#70563, plant#70564, first(calculationDateTime#70565)()#69321 AS calculationDateTime#70640, collect_list(stockQuantity#70639, 0, 0)#69322 AS stockQuantities#70641]
(698) Filter
Input [4]: [product#70563, plant#70564, calculationDateTime#70640, stockQuantities#70641]
Condition : ((size(stockQuantities#70641, true) > 0) AND isnotnull(calculationDateTime#70640))
(699) Generate
Input [4]: [product#70563, plant#70564, calculationDateTime#70640, stockQuantities#70641]
Arguments: explode(stockQuantities#70641), [product#70563, plant#70564, calculationDateTime#70640], false, [stockQuantity#70642]
(700) Filter
Input [4]: [product#70563, plant#70564, calculationDateTime#70640, stockQuantity#70642]
Condition : (((isnotnull(stockQuantity#70642.specialStockIndicator.code) AND isnotnull(stockQuantity#70642.stockType.code)) AND (stockQuantity#70642.specialStockIndicator.code = )) AND (stockQuantity#70642.stockType.code = 01))
(701) Project
Output [4]: [plant#70564.internalRefUUID AS plantUuid#70644, product#70563.internalRefUUID AS productUuid#70646, calculationDateTime#70640, stockQuantity#70642.storagelocation.id AS storageLocId#70647]
Input [4]: [product#70563, plant#70564, calculationDateTime#70640, stockQuantity#70642]
(702) Exchange
Input [4]: [plantUuid#70644, productUuid#70646, calculationDateTime#70640, storageLocId#70647]
Arguments: hashpartitioning(productUuid#70646, plantUuid#70644, 25), ENSURE_REQUIREMENTS, [plan_id=32553]
(703) Sort
Input [4]: [plantUuid#70644, productUuid#70646, calculationDateTime#70640, storageLocId#70647]
Arguments: [productUuid#70646 ASC NULLS FIRST, plantUuid#70644 ASC NULLS FIRST], false, 0
(704) Scan parquet
Output [3]: [product#70652, plant#70653, orderDateTime#70654]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(705) Filter
Input [3]: [product#70652, plant#70653, orderDateTime#70654]
Condition : ((isnotnull(orderDateTime#70654) AND isnotnull(plant#70653)) AND isnotnull(product#70652))
(706) Exchange
Input [3]: [product#70652, plant#70653, orderDateTime#70654]
Arguments: hashpartitioning(product#70652, plant#70653, 25), REPARTITION_BY_NUM, [plan_id=31926]
(707) Scan parquet
Output [4]: [plant#70658, product#70659, productionAspect#70661, systemId#70656]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(708) Project
Output [4]: [plant#70658.internalRefUUID AS _extract_internalRefUUID#71415, product#70659.internalRefUUID AS _extract_internalRefUUID#71418, productionAspect#70661.productMovementPlants.unitOfIssue.code AS _extract_code#71416, productionAspect#70661.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71417]
Input [4]: [plant#70658, product#70659, productionAspect#70661, systemId#70656]
(709) Exchange
Input [4]: [_extract_internalRefUUID#71415, _extract_internalRefUUID#71418, _extract_code#71416, _extract_internalRefUUID#71417]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31251]
(710) Project
Output [4]: [_extract_code#71416, _extract_internalRefUUID#71417, _extract_internalRefUUID#71418, _extract_internalRefUUID#71415]
Input [4]: [_extract_internalRefUUID#71415, _extract_internalRefUUID#71418, _extract_code#71416, _extract_internalRefUUID#71417]
(711) BroadcastExchange
Input [4]: [_extract_code#71416, _extract_internalRefUUID#71417, _extract_internalRefUUID#71418, _extract_internalRefUUID#71415]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32520]
(712) BroadcastHashJoin
Left keys [4]: [coalesce(product#70652, ), isnull(product#70652), coalesce(plant#70653, ), isnull(plant#70653)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71418, ), isnull(_extract_internalRefUUID#71418), coalesce(_extract_internalRefUUID#71415, ), isnull(_extract_internalRefUUID#71415)]
Join type: LeftOuter
Join condition: None
(713) Project
Output [5]: [product#70652, plant#70653, orderDateTime#70654, _extract_code#71416, _extract_internalRefUUID#71417]
Input [7]: [product#70652, plant#70653, orderDateTime#70654, _extract_code#71416, _extract_internalRefUUID#71417, _extract_internalRefUUID#71418, _extract_internalRefUUID#71415]
(714) Scan parquet
Output [2]: [internalUUID#70666, systemId#70665]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(715) Project
Output [1]: [internalUUID#70666]
Input [2]: [internalUUID#70666, systemId#70665]
(716) Exchange
Input [1]: [internalUUID#70666]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31257]
(717) BroadcastExchange
Input [1]: [internalUUID#70666]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32528]
(718) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70653, ), isnull(plant#70653)]
Right keys [2]: [coalesce(internalUUID#70666, ), isnull(internalUUID#70666)]
Join type: LeftOuter
Join condition: None
(719) Project
Output [5]: [product#70652, plant#70653, orderDateTime#70654, _extract_code#71416, _extract_internalRefUUID#71417]
Input [6]: [product#70652, plant#70653, orderDateTime#70654, _extract_code#71416, _extract_internalRefUUID#71417, internalUUID#70666]
(720) Scan parquet
Output [4]: [internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686, systemId#70679]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(721) Filter
Input [4]: [internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686, systemId#70679]
Condition : isnotnull(internalUUID#70680)
(722) Project
Output [3]: [internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686]
Input [4]: [internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686, systemId#70679]
(723) Exchange
Input [3]: [internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31267]
(724) BroadcastExchange
Input [3]: [internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32536]
(725) BroadcastHashJoin
Left keys [1]: [product#70652]
Right keys [1]: [internalUUID#70680]
Join type: LeftOuter
Join condition: None
(726) Project
Output [4]: [product#70652, plant#70653, orderDateTime#70654, filter(unitOfMeasureConversions#70686, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71416) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71417)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70684.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70684.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70691]
Input [8]: [product#70652, plant#70653, orderDateTime#70654, _extract_code#71416, _extract_internalRefUUID#71417, internalUUID#70680, baseUnitOfMeasure#70684, unitOfMeasureConversions#70686]
(727) Generate
Input [4]: [product#70652, plant#70653, orderDateTime#70654, filteredUnitOfMeasureConversions#70691]
Arguments: explode(filteredUnitOfMeasureConversions#70691.quantityNumerator), [product#70652, plant#70653, orderDateTime#70654, filteredUnitOfMeasureConversions#70691], true, [quantityNumerator#70692]
(728) Project
Output [4]: [product#70652, plant#70653, orderDateTime#70654, filteredUnitOfMeasureConversions#70691.quantityDenominator AS _extract_quantityDenominator#71051]
Input [5]: [product#70652, plant#70653, orderDateTime#70654, filteredUnitOfMeasureConversions#70691, quantityNumerator#70692]
(729) Generate
Input [4]: [product#70652, plant#70653, orderDateTime#70654, _extract_quantityDenominator#71051]
Arguments: explode(_extract_quantityDenominator#71051), [product#70652, plant#70653, orderDateTime#70654], true, [quantityDenominator#70693]
(730) Project
Output [3]: [product#70652, plant#70653, orderDateTime#70654]
Input [4]: [product#70652, plant#70653, orderDateTime#70654, quantityDenominator#70693]
(731) Sort
Input [3]: [product#70652, plant#70653, orderDateTime#70654]
Arguments: [product#70652 ASC NULLS FIRST, plant#70653 ASC NULLS FIRST], false, 0
(732) SortMergeJoin
Left keys [2]: [productUuid#70646, plantUuid#70644]
Right keys [2]: [product#70652, plant#70653]
Join type: Inner
Join condition: (calculationDateTime#70640 <= orderDateTime#70654)
(733) Project
Output [4]: [plantUuid#70644 AS plant#70696, productUuid#70646 AS product#70697, storageLocId#70647, calculationDateTime#70640]
Input [7]: [plantUuid#70644, productUuid#70646, calculationDateTime#70640, storageLocId#70647, product#70652, plant#70653, orderDateTime#70654]
(734) HashAggregate
Input [4]: [plant#70696, product#70697, storageLocId#70647, calculationDateTime#70640]
Keys [3]: [plant#70696, product#70697, storageLocId#70647]
Functions [1]: [partial_max(calculationDateTime#70640)]
Aggregate Attributes [1]: [max#73960]
Results [4]: [plant#70696, product#70697, storageLocId#70647, max#73961]
(735) HashAggregate
Input [4]: [plant#70696, product#70697, storageLocId#70647, max#73961]
Keys [3]: [plant#70696, product#70697, storageLocId#70647]
Functions [1]: [max(calculationDateTime#70640)]
Aggregate Attributes [1]: [max(calculationDateTime#70640)#69456]
Results [4]: [plant#70696, product#70697, storageLocId#70647, max(calculationDateTime#70640)#69456 AS max_calc_datetime#70698]
(736) Exchange
Input [4]: [plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
Arguments: hashpartitioning(coalesce(plant#70696, ), isnull(plant#70696), coalesce(product#70697, ), isnull(product#70697), coalesce(storageLocId#70647, ), isnull(storageLocId#70647), coalesce(max_calc_datetime#70698, 1970-01-01 00:00:00), isnull(max_calc_datetime#70698), 25), ENSURE_REQUIREMENTS, [plan_id=32570]
(737) Sort
Input [4]: [plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
Arguments: [coalesce(plant#70696, ) ASC NULLS FIRST, isnull(plant#70696) ASC NULLS FIRST, coalesce(product#70697, ) ASC NULLS FIRST, isnull(product#70697) ASC NULLS FIRST, coalesce(storageLocId#70647, ) ASC NULLS FIRST, isnull(storageLocId#70647) ASC NULLS FIRST, coalesce(max_calc_datetime#70698, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(max_calc_datetime#70698) ASC NULLS FIRST], false, 0
(738) SortMergeJoin
Left keys [8]: [coalesce(plant#70559, ), isnull(plant#70559), coalesce(product#70560, ), isnull(product#70560), coalesce(storageLocId#70506, ), isnull(storageLocId#70506), coalesce(calculationDateTime#70499, 1970-01-01 00:00:00), isnull(calculationDateTime#70499)]
Right keys [8]: [coalesce(plant#70696, ), isnull(plant#70696), coalesce(product#70697, ), isnull(product#70697), coalesce(storageLocId#70647, ), isnull(storageLocId#70647), coalesce(max_calc_datetime#70698, 1970-01-01 00:00:00), isnull(max_calc_datetime#70698)]
Join type: Inner
Join condition: None
(739) Project
Output [4]: [plant#70559, product#70560, calculationDateTime#70499, quantity#70508]
Input [9]: [plant#70559, product#70560, storageLocId#70506, calculationDateTime#70499, quantity#70508, plant#70696, product#70697, storageLocId#70647, max_calc_datetime#70698]
(740) HashAggregate
Input [4]: [plant#70559, product#70560, calculationDateTime#70499, quantity#70508]
Keys [2]: [product#70560, plant#70559]
Functions [2]: [partial_min(calculationDateTime#70499), partial_sum(quantity#70508)]
Aggregate Attributes [2]: [min#73942, sum#73943]
Results [4]: [product#70560, plant#70559, min#73944, sum#73945]
(741) Exchange
Input [4]: [product#70560, plant#70559, min#73944, sum#73945]
Arguments: hashpartitioning(product#70560, plant#70559, 25), ENSURE_REQUIREMENTS, [plan_id=32580]
(742) HashAggregate
Input [4]: [product#70560, plant#70559, min#73944, sum#73945]
Keys [2]: [product#70560, plant#70559]
Functions [2]: [min(calculationDateTime#70499), sum(quantity#70508)]
Aggregate Attributes [2]: [min(calculationDateTime#70499)#69768, sum(quantity#70508)#69767]
Results [4]: [product#70560, plant#70559, min(calculationDateTime#70499)#69768 AS minCalculationDateTime#70700, round(sum(quantity#70508)#69767, 0) AS roundedQuantitySum#70701]
(743) Sort
Input [4]: [product#70560, plant#70559, minCalculationDateTime#70700, roundedQuantitySum#70701]
Arguments: [product#70560 ASC NULLS FIRST, plant#70559 ASC NULLS FIRST], false, 0
(744) SortMergeJoin
Left keys [2]: [product#70356, plant#70357]
Right keys [2]: [product#70560, plant#70559]
Join type: LeftOuter
Join condition: None
(745) Project
Output [8]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, CASE WHEN isnotnull(minCalculationDateTime#70700) THEN minCalculationDateTime#70700 ELSE orderDateTime#70358 END AS stockDatetime#70702, CASE WHEN isnotnull(roundedQuantitySum#70701) THEN array(roundedQuantitySum#70701) ELSE [0.0] END AS stockBatchQuantities#70703]
Input [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, product#70560, plant#70559, minCalculationDateTime#70700, roundedQuantitySum#70701]
(746) Filter
Input [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
Condition : (isnotnull(product#69898) AND isnotnull(plant#69899))
(747) BroadcastExchange
Input [9]: [product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
Arguments: HashedRelationBroadcastMode(List(input[0, string, false], input[1, string, false]),false), [plan_id=32601]
(748) BroadcastHashJoin
Left keys [2]: [product#70356, plant#70357]
Right keys [2]: [product#69898, plant#69899]
Join type: LeftOuter
Join condition: None
(749) Project
Output [9]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, CASE WHEN isnotnull(product#69898) THEN struct(demandChannel, demandChannel#69900, demandStream, demandStream#69901, considerVariance, considerVariance#69902, demandTimeBuckets, demandTimeBuckets#69903, demandPointInTimeStart, demandPointInTimeStart#69904, demandPointInTimeEnd, demandPointInTimeEnd#69905, demandPointInTime, demandPointInTime#69906) END AS rawDemands#69977]
Input [17]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, product#69898, plant#69899, demandChannel#69900, demandStream#69901, considerVariance#69902, demandTimeBuckets#69903, demandPointInTimeStart#69904, demandPointInTimeEnd#69905, demandPointInTime#69906]
(750) ObjectHashAggregate
Input [9]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, rawDemands#69977]
Keys [9]: [product#70356, plant#70357, orderDateTime#70358, knownfloatingpointnormalized(if (isnull(productPlant#70368)) null else named_struct(systemId, productPlant#70368.systemId, internalUUID, productPlant#70368.internalUUID, plant, productPlant#70368.plant, product, productPlant#70368.product, blockedForReplenishmentStartingFrom, productPlant#70368.blockedForReplenishmentStartingFrom, productionAspect, knownfloatingpointnormalized(if (isnull(productPlant#70368.productionAspect)) null else named_struct(productMovementPlants, productPlant#70368.productionAspect.productMovementPlants, productPlanningPlants, knownfloatingpointnormalized(if (isnull(productPlant#70368.productionAspect.productPlanningPlants)) null else named_struct(mrpType, productPlant#70368.productionAspect.productPlanningPlants.mrpType, abcIndicator, productPlant#70368.productionAspect.productPlanningPlants.abcIndicator, mrpController, productPlant#70368.productionAspect.productPlanningPlants.mrpController, minimumLotSize, knownfloatingpointnormalized(normalizenanandzero(productPlant#70368.productionAspect.productPlanningPlants.minimumLotSize)), minimumSafetyStock, knownfloatingpointnormalized(normalizenanandzero(productPlant#70368.productionAspect.productPlanningPlants.minimumSafetyStock)), logisticalRoundingProfile, productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile, sourceOfSupplyCategory, productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory)))), salesPlant, productPlant#70368.salesPlant, listing, productPlant#70368.listing)) AS productPlant#70368, timezonecode#70382, knownfloatingpointnormalized(normalizenanandzero(outboundUnit#70401)) AS outboundUnit#70401, stockDatetime#70702, knownfloatingpointnormalized(transform(stockBatchQuantities#70703, lambdafunction(knownfloatingpointnormalized(normalizenanandzero(lambda arg#73933)), lambda arg#73933, false))) AS stockBatchQuantities#70703, [7226582400000000] AS [7226582400000000]#73932]
Functions [1]: [partial_collect_list(rawDemands#69977, 0, 0)]
Aggregate Attributes [1]: [buf#73940]
Results [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000]#73932, buf#73941]
(751) ObjectHashAggregate
Input [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000]#73932, buf#73941]
Keys [9]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000]#73932]
Functions [1]: [collect_list(rawDemands#69977, 0, 0)]
Aggregate Attributes [1]: [collect_list(rawDemands#69977, 0, 0)#70170]
Results [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, [7226582400000000] AS stockBatchShelfLifeEndDatetimes#70704, collect_list(rawDemands#69977, 0, 0)#70170 AS rawDemands#70004]
(752) DeserializeToObject
Input [10]: [product#70356, plant#70357, orderDateTime#70358, productPlant#70368, timezonecode#70382, outboundUnit#70401, stockDatetime#70702, stockBatchQuantities#70703, stockBatchShelfLifeEndDatetimes#70704, rawDemands#70004]
Arguments: createexternalrow(invoke(product#70356.toString()), invoke(plant#70357.toString()), static_invoke(DateTimeUtils.toJavaTimestamp(orderDateTime#70358)), if (isnull(productPlant#70368)) null else createexternalrow(invoke(productPlant#70368.systemId.toString()), invoke(productPlant#70368.internalUUID.toString()), if (isnull(productPlant#70368.plant)) null else createexternalrow(invoke(productPlant#70368.plant.internalRefUUID.toString()), invoke(productPlant#70368.plant.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), if (isnull(productPlant#70368.product)) null else createexternalrow(invoke(productPlant#70368.product.internalRefUUID.toString()), invoke(productPlant#70368.product.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(DateTimeUtils.toJavaDate(productPlant#70368.blockedForReplenishmentStartingFrom)), if (isnull(productPlant#70368.productionAspect)) null else createexternalrow(if (isnull(productPlant#70368.productionAspect.productMovementPlants)) null else createexternalrow(if (isnull(productPlant#70368.productionAspect.productMovementPlants.unitOfIssue)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productMovementPlants.unitOfIssue.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants)) null else createexternalrow(if (isnull(productPlant#70368.productionAspect.productPlanningPlants.mrpType)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpType.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpType.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.abcIndicator)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.abcIndicator.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.abcIndicator.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.mrpController)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpController.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.mrpController.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(java.lang.Double.valueOf(productPlant#70368.productionAspect.productPlanningPlants.minimumLotSize)), static_invoke(java.lang.Double.valueOf(productPlant#70368.productionAspect.productPlanningPlants.minimumSafetyStock)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.logisticalRoundingProfile.logisticalRoundingProfile.toString()), StructField(internalRefUUID,StringType,true), StructField(logisticalRoundingProfile,StringType,true)), if (isnull(productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory)) null else createexternalrow(invoke(productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory.internalRefUUID.toString()), invoke(productPlant#70368.productionAspect.productPlanningPlants.sourceOfSupplyCategory.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(minimumLotSize,DoubleType,true), StructField(minimumSafetyStock,DoubleType,true), StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true), StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)), if (isnull(productPlant#70368.salesPlant)) null else createexternalrow(if (isnull(productPlant#70368.salesPlant.loadingGroup)) null else createexternalrow(invoke(productPlant#70368.salesPlant.loadingGroup.internalRefUUID.toString()), invoke(productPlant#70368.salesPlant.loadingGroup.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), mapobjects(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1), if (isnull(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validFrom)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validTo)), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).isListed)), StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true)), productPlant#70368.listing, Some(class scala.collection.mutable.ArraySeq)), StructField(systemId,StringType,true), StructField(internalUUID,StringType,true), StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(blockedForReplenishmentStartingFrom,DateType,true), StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true), StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)), invoke(timezonecode#70382.toString()), static_invoke(java.lang.Double.valueOf(outboundUnit#70401)), static_invoke(DateTimeUtils.toJavaTimestamp(stockDatetime#70702)), mapobjects(lambdavariable(MapObject, DoubleType, true, -2), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, DoubleType, true, -2))), stockBatchQuantities#70703, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, TimestampType, false, -3), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, TimestampType, false, -3))), stockBatchShelfLifeEndDatetimes#70704, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4), createexternalrow(invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandChannel.toString()), invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandStream.toString()), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).considerVariance)), mapobjects(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5), if (isnull(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandStartDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandVariance)), StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandTimeBuckets, Some(class scala.collection.mutable.ArraySeq)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeStart)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeEnd)), mapobjects(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6), if (isnull(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandVariance)), StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTime, Some(class scala.collection.mutable.ArraySeq)), StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)), rawDemands#70004, Some(class scala.collection.mutable.ArraySeq)), StructField(product,StringType,true), StructField(plant,StringType,true), StructField(orderDateTime,TimestampType,true), StructField(productPlant,StructType(StructField(systemId,StringType,true),StructField(internalUUID,StringType,true),StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(blockedForReplenishmentStartingFrom,DateType,true),StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true),StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)),true), StructField(timezonecode,StringType,true), StructField(outboundUnit,DoubleType,true), StructField(stockDatetime,TimestampType,true), StructField(stockBatchQuantities,ArrayType(DoubleType,true),false), StructField(stockBatchShelfLifeEndDatetimes,ArrayType(TimestampType,false),false), StructField(rawDemands,ArrayType(StructType(StructField(demandChannel,StringType,true),StructField(demandStream,StringType,true),StructField(considerVariance,BooleanType,true),StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true),StructField(demandPointInTimeStart,TimestampType,true),StructField(demandPointInTimeEnd,TimestampType,true),StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)),false),false)), obj#70342: org.apache.spark.sql.Row
(753) MapElements
Input [1]: [obj#70342]
Arguments: com.sap.s4hana.eureka.business.crporderquantitykpiservice.core.business.datasnapshot.DemandTransformatorImpl$$Lambda/0x00000008024b0d88@22b451ac, obj#70347: org.apache.spark.sql.Row
(754) SerializeFromObject
Input [1]: [obj#70347]
Arguments: [if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(0))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, product), StringType, ObjectType(class java.lang.String)))) AS product#70352, if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(1))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, plant), StringType, ObjectType(class java.lang.String)))) AS plant#70353]
(755) Exchange
Input [2]: [product#70352, plant#70353]
Arguments: hashpartitioning(coalesce(plant#70353, ), isnull(plant#70353), coalesce(product#70352, ), isnull(product#70352), 25), ENSURE_REQUIREMENTS, [plan_id=32621]
(756) Sort
Input [2]: [product#70352, plant#70353]
Arguments: [coalesce(plant#70353, ) ASC NULLS FIRST, isnull(plant#70353) ASC NULLS FIRST, coalesce(product#70352, ) ASC NULLS FIRST, isnull(product#70352) ASC NULLS FIRST], false, 0
(757) SortMergeJoin
Left keys [4]: [coalesce(plant#69016, ), isnull(plant#69016), coalesce(product#69015, ), isnull(product#69015)]
Right keys [4]: [coalesce(plant#70353, ), isnull(plant#70353), coalesce(product#70352, ), isnull(product#70352)]
Join type: LeftOuter
Join condition: None
(758) Project
Output [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Input [6]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867, product#70352, plant#70353]
(759) Exchange
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: hashpartitioning(plant#69016, 25), ENSURE_REQUIREMENTS, [plan_id=32684]
(760) Sort
Input [4]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867]
Arguments: [plant#69016 ASC NULLS FIRST], false, 0
(761) Scan parquet
Output [2]: [marketUnit#70831, distributionCenters#70837]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(762) Filter
Input [2]: [marketUnit#70831, distributionCenters#70837]
Condition : (((marketUnit#70831 <=> Daily_grocery) AND (size(distributionCenters#70837, true) > 0)) AND isnotnull(distributionCenters#70837))
(763) Project
Output [1]: [distributionCenters#70837]
Input [2]: [marketUnit#70831, distributionCenters#70837]
(764) Exchange
Input [1]: [distributionCenters#70837]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31353]
(765) Generate
Input [1]: [distributionCenters#70837]
Arguments: explode(distributionCenters#70837), false, [distributionCenters#70853]
(766) Filter
Input [1]: [distributionCenters#70853]
Condition : (((size(distributionCenters#70853.storageLocations.internalRefUUID, true) > 0) AND isnotnull(distributionCenters#70853.storageLocations.internalRefUUID)) AND isnotnull(distributionCenters#70853.internalRefUUID))
(767) Project
Output [2]: [distributionCenters#70853.internalRefUUID AS _extract_internalRefUUID#71059, distributionCenters#70853.storageLocations.internalRefUUID AS _extract_internalRefUUID#71430]
Input [1]: [distributionCenters#70853]
(768) Generate
Input [2]: [_extract_internalRefUUID#71059, _extract_internalRefUUID#71430]
Arguments: explode(_extract_internalRefUUID#71430), [_extract_internalRefUUID#71059], false, [storageLocations#70856]
(769) Filter
Input [2]: [_extract_internalRefUUID#71059, storageLocations#70856]
Condition : isnotnull(storageLocations#70856)
(770) Project
Output [2]: [_extract_internalRefUUID#71059 AS plantUuid#70860, storageLocations#70856 AS storageLocUuid#70864]
Input [2]: [_extract_internalRefUUID#71059, storageLocations#70856]
(771) BroadcastExchange
Input [2]: [plantUuid#70860, storageLocUuid#70864]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32671]
(772) Scan parquet
Output [2]: [product#70869, plant#70870]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002107/shardId=0_1_10000002107]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(773) Filter
Input [2]: [product#70869, plant#70870]
Condition : isnotnull(plant#70870)
(774) Exchange
Input [2]: [product#70869, plant#70870]
Arguments: hashpartitioning(product#70869, plant#70870, 25), REPARTITION_BY_NUM, [plan_id=32005]
(775) Scan parquet
Output [4]: [plant#70875, product#70876, productionAspect#70878, systemId#70873]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(776) Project
Output [4]: [plant#70875.internalRefUUID AS _extract_internalRefUUID#71438, product#70876.internalRefUUID AS _extract_internalRefUUID#71435, productionAspect#70878.productMovementPlants.unitOfIssue.code AS _extract_code#71436, productionAspect#70878.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71437]
Input [4]: [plant#70875, product#70876, productionAspect#70878, systemId#70873]
(777) Exchange
Input [4]: [_extract_internalRefUUID#71438, _extract_internalRefUUID#71435, _extract_code#71436, _extract_internalRefUUID#71437]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31370]
(778) Project
Output [4]: [_extract_code#71436, _extract_internalRefUUID#71437, _extract_internalRefUUID#71435, _extract_internalRefUUID#71438]
Input [4]: [_extract_internalRefUUID#71438, _extract_internalRefUUID#71435, _extract_code#71436, _extract_internalRefUUID#71437]
(779) BroadcastExchange
Input [4]: [_extract_code#71436, _extract_internalRefUUID#71437, _extract_internalRefUUID#71435, _extract_internalRefUUID#71438]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32634]
(780) BroadcastHashJoin
Left keys [4]: [coalesce(product#70869, ), isnull(product#70869), coalesce(plant#70870, ), isnull(plant#70870)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71435, ), isnull(_extract_internalRefUUID#71435), coalesce(_extract_internalRefUUID#71438, ), isnull(_extract_internalRefUUID#71438)]
Join type: LeftOuter
Join condition: None
(781) Project
Output [4]: [product#70869, plant#70870, _extract_code#71436, _extract_internalRefUUID#71437]
Input [6]: [product#70869, plant#70870, _extract_code#71436, _extract_internalRefUUID#71437, _extract_internalRefUUID#71435, _extract_internalRefUUID#71438]
(782) Scan parquet
Output [2]: [internalUUID#70905, systemId#70904]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(783) Project
Output [1]: [internalUUID#70905]
Input [2]: [internalUUID#70905, systemId#70904]
(784) Exchange
Input [1]: [internalUUID#70905]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31382]
(785) BroadcastExchange
Input [1]: [internalUUID#70905]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32642]
(786) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70870, ), isnull(plant#70870)]
Right keys [2]: [coalesce(internalUUID#70905, ), isnull(internalUUID#70905)]
Join type: LeftOuter
Join condition: None
(787) Project
Output [4]: [product#70869, plant#70870, _extract_code#71436, _extract_internalRefUUID#71437]
Input [5]: [product#70869, plant#70870, _extract_code#71436, _extract_internalRefUUID#71437, internalUUID#70905]
(788) Scan parquet
Output [4]: [internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938, systemId#70931]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(789) Filter
Input [4]: [internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938, systemId#70931]
Condition : isnotnull(internalUUID#70932)
(790) Project
Output [3]: [internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938]
Input [4]: [internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938, systemId#70931]
(791) Exchange
Input [3]: [internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31391]
(792) BroadcastExchange
Input [3]: [internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32646]
(793) BroadcastHashJoin
Left keys [1]: [product#70869]
Right keys [1]: [internalUUID#70932]
Join type: LeftOuter
Join condition: None
(794) Project
Output [2]: [plant#70870, filter(unitOfMeasureConversions#70938, lambdafunction((((lambda u#69110.measurementUnit1.code = _extract_code#71436) AND (lambda u#69110.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71437)) AND ((lambda u#69110.measurementUnit2.code = baseUnitOfMeasure#70936.code) AND (lambda u#69110.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70936.internalRefUUID))), lambda u#69110, false)) AS filteredUnitOfMeasureConversions#70945]
Input [7]: [product#70869, plant#70870, _extract_code#71436, _extract_internalRefUUID#71437, internalUUID#70932, baseUnitOfMeasure#70936, unitOfMeasureConversions#70938]
(795) Generate
Input [2]: [plant#70870, filteredUnitOfMeasureConversions#70945]
Arguments: explode(filteredUnitOfMeasureConversions#70945.quantityNumerator), [plant#70870, filteredUnitOfMeasureConversions#70945], true, [quantityNumerator#70947]
(796) Project
Output [2]: [plant#70870, filteredUnitOfMeasureConversions#70945.quantityDenominator AS _extract_quantityDenominator#71063]
Input [3]: [plant#70870, filteredUnitOfMeasureConversions#70945, quantityNumerator#70947]
(797) Generate
Input [2]: [plant#70870, _extract_quantityDenominator#71063]
Arguments: explode(_extract_quantityDenominator#71063), [plant#70870], true, [quantityDenominator#70948]
(798) Project
Output [1]: [plant#70870]
Input [2]: [plant#70870, quantityDenominator#70948]
(799) HashAggregate
Input [1]: [plant#70870]
Keys [1]: [plant#70870]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70870]
(800) Exchange
Input [1]: [plant#70870]
Arguments: hashpartitioning(plant#70870, 25), ENSURE_REQUIREMENTS, [plan_id=32667]
(801) HashAggregate
Input [1]: [plant#70870]
Keys [1]: [plant#70870]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70870]
(802) BroadcastHashJoin
Left keys [1]: [plantUuid#70860]
Right keys [1]: [plant#70870]
Join type: Inner
Join condition: None
(803) Project
Output [2]: [plantUuid#70860 AS plant#70808, storageLocUuid#70864]
Input [3]: [plantUuid#70860, storageLocUuid#70864, plant#70870]
(804) Sort
Input [2]: [plant#70808, storageLocUuid#70864]
Arguments: [plant#70808 ASC NULLS FIRST], false, 0
(805) SortMergeJoin
Left keys [1]: [plant#69016]
Right keys [1]: [plant#70808]
Join type: Inner
Join condition: None
(806) Project
Output [5]: [plant#69016, product#69015, timezonecode#69080, stockDatetime#69867, storageLocUuid#70864]
Input [6]: [product#69015, plant#69016, timezonecode#69080, stockDatetime#69867, plant#70808, storageLocUuid#70864]
(807) Filter
Input [6]: [internalUUID#70739, id#70740, supplier#70742, isMarkedForDeletion#70745, items#70746, systemId#70738]
Condition : (((isnotnull(isMarkedForDeletion#70745) AND NOT isMarkedForDeletion#70745) AND (size(items#70746, true) > 0)) AND isnotnull(items#70746))
(808) Project
Output [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Input [6]: [internalUUID#70739, id#70740, supplier#70742, isMarkedForDeletion#70745, items#70746, systemId#70738]
(809) Exchange
Input [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31424]
(810) Generate
Input [4]: [internalUUID#70739, id#70740, supplier#70742, items#70746]
Arguments: explode(items#70746), [internalUUID#70739, id#70740, supplier#70742], false, [item#70754]
(811) Filter
Input [4]: [internalUUID#70739, id#70740, supplier#70742, item#70754]
Condition : ((((isnotnull(item#70754.deletionIndicator) AND isnotnull(item#70754.completeIndicator)) AND isnotnull(item#70754.returnIndicator)) AND ((NOT item#70754.deletionIndicator AND NOT item#70754.completeIndicator) AND NOT item#70754.returnIndicator)) AND ((isnotnull(item#70754.product.internalRefUUID) AND isnotnull(item#70754.plant.internalRefUUID)) AND isnotnull(item#70754.storageLocation.internalRefUUID)))
(812) Project
Output [14]: [id#70740, internalUUID#70739, supplier#70742.internalRefUUID AS supplierUuid#70749, item#70754.plant.internalRefUUID AS plant#70763, item#70754.product.internalRefUUID AS product#70764, item#70754.plant.id AS externalPlantId#70765, item#70754.product.id AS externalProductId#70766, item#70754.storageLocation.internalRefUUID AS storageLocUuid#70767, item#70754.quantityUnit.code AS quantityUom#70768, item#70754.quantityUnit.internalRefUUID AS quantityUomUuid#70769, item#70754.quantityNumerator AS quantityNumerator#70771, item#70754.quantityDenominator AS quantityDenominator#70772, item#70754.scheduleLines AS scheduleLines#70774, item#70754.confirmations AS confirmations#70776]
Input [4]: [internalUUID#70739, id#70740, supplier#70742, item#70754]
(813) BroadcastExchange
Input [14]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776]
Arguments: HashedRelationBroadcastMode(List(input[4, string, true], input[3, string, true], input[7, string, true]),false), [plan_id=32690]
(814) BroadcastHashJoin
Left keys [3]: [product#69015, plant#69016, storageLocUuid#70864]
Right keys [3]: [product#70764, plant#70763, storageLocUuid#70767]
Join type: Inner
Join condition: None
(815) Project
Output [17]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776, stockDatetime#69867, timezonecode#69080, filter(confirmations#70776, lambdafunction(((NOT lambda x_1#70965.deletionIndicator AND lambda x_1#70965.isRelevantForConfirmation) AND isnotnull(lambda x_1#70965.confirmedDeliveryAt)), lambda x_1#70965, false)) AS validConfirmations#70964]
Input [19]: [plant#69016, product#69015, timezonecode#69080, stockDatetime#69867, storageLocUuid#70864, id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776]
(816) AdaptiveSparkPlan
Output [17]: [id#70740, internalUUID#70739, supplierUuid#70749, plant#70763, product#70764, externalPlantId#70765, externalProductId#70766, storageLocUuid#70767, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, confirmations#70776, stockDatetime#69867, timezonecode#69080, validConfirmations#70964]
Arguments: isFinalPlan=true
(817) Project
Output [11]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, validConfirmations#70964, size(validConfirmations#70964, false) AS _w0#75047]
Input [10]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityDenominator#70772, quantityNumerator#70771, quantityUom#70768, quantityUomUuid#70769, scheduleLines#70774, validConfirmations#70964]
(818) Exchange
Input [11]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, validConfirmations#70964, _w0#75047]
Arguments: hashpartitioning(internalUUID#70739, 25), ENSURE_REQUIREMENTS, [plan_id=32756]
(819) Sort
Input [11]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, validConfirmations#70964, _w0#75047]
Arguments: [internalUUID#70739 ASC NULLS FIRST], false, 0
(820) Window
Input [11]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, validConfirmations#70964, _w0#75047]
Arguments: [sum(_w0#75047) windowspecdefinition(internalUUID#70739, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#75048L], [internalUUID#70739]
(821) Project
Output [3]: [plant#70763, product#70764, filter(transform(CASE WHEN (_we0#75048L > 0) THEN transform(validConfirmations#70964, lambdafunction(struct(openQty, greatest((lambda x_12#75038.confirmedQuantity - coalesce(lambda x_12#75038.mrpRelevantQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_12#75038.confirmedDeliveryAt), lambda x_12#75038, false)) ELSE transform(scheduleLines#70774, lambdafunction(struct(openQty, greatest((lambda x_11#75040.quantity - coalesce(lambda x_11#75040.receivedQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_11#75040.deliveryAt), lambda x_11#75040, false)) END, lambdafunction(struct(documentId, id#70740, documentUuid, internalUUID#70739, openQty, lambda x_13#75042.openQty, openQtyInBaseUnit, ((lambda x_13#75042.openQty * quantityNumerator#70771) / quantityDenominator#70772), qtyUom, quantityUom#70768, qtyUomUuid, quantityUomUuid#70769, availabilityDatetime, lambda x_13#75042.availabilityDatetime, shelfLifeEndDatetime, 2199-01-01 00:00:00), lambda x_13#75042, false)), lambdafunction(NOT (lambda x_14#75044.openQty = 0.0), lambda x_14#75044, false)) AS openGoodsMovements#75024]
Input [12]: [id#70740, internalUUID#70739, plant#70763, product#70764, quantityUom#70768, quantityUomUuid#70769, quantityNumerator#70771, quantityDenominator#70772, scheduleLines#70774, validConfirmations#70964, _w0#75047, _we0#75048L]
(822) ObjectHashAggregate
Input [3]: [plant#70763, product#70764, openGoodsMovements#75024]
Keys [2]: [product#70764, plant#70763]
Functions [1]: [partial_collect_list(openGoodsMovements#75024, 0, 0)]
Aggregate Attributes [1]: [buf#75602]
Results [3]: [product#70764, plant#70763, buf#75603]
(823) Exchange
Input [3]: [product#70764, plant#70763, buf#75603]
Arguments: hashpartitioning(product#70764, plant#70763, 25), ENSURE_REQUIREMENTS, [plan_id=32769]
(824) ObjectHashAggregate
Input [3]: [product#70764, plant#70763, buf#75603]
Keys [2]: [product#70764, plant#70763]
Functions [1]: [collect_list(openGoodsMovements#75024, 0, 0)]
Aggregate Attributes [1]: [collect_list(openGoodsMovements#75024, 0, 0)#75089]
Results [3]: [product#70764, plant#70763, flatten(collect_list(openGoodsMovements#75024, 0, 0)#75089) AS openGoodsMovements#75051]
(825) Filter
Input [3]: [product#70764, plant#70763, openGoodsMovements#75051]
Condition : (size(openGoodsMovements#75051, false) > 0)
(826) AdaptiveSparkPlan
Output [3]: [product#70764, plant#70763, openGoodsMovements#75051]
Arguments: isFinalPlan=true
(827) TableCacheQueryStage
Output [3]: [openGoodsMovements#75051, plant#70763, product#70764]
Arguments: 0
(828) Filter
Input [3]: [openGoodsMovements#75051, plant#70763, product#70764]
Condition : (size(transform(openGoodsMovements#75051, lambdafunction(struct(documentType, PO, documentId, lambda x_28#90539.documentId, documentUuid, lambda x_28#90539.documentUuid, openQty, lambda x_28#90539.openQty, qtyUomUuid, lambda x_28#90539.qtyUomUuid, qtyUom, lambda x_28#90539.qtyUom, plannedDeliveryDatetime, lambda x_28#90539.availabilityDatetime), lambda x_28#90539, false)), false) > 0)
(829) Project
Output [4]: [product#70764, plant#70763 AS location#90535, transform(openGoodsMovements#75051, lambdafunction(struct(documentType, PO, documentId, lambda x_28#90539.documentId, documentUuid, lambda x_28#90539.documentUuid, openQty, lambda x_28#90539.openQty, qtyUomUuid, lambda x_28#90539.qtyUomUuid, qtyUom, lambda x_28#90539.qtyUom, plannedDeliveryDatetime, lambda x_28#90539.availabilityDatetime), lambda x_28#90539, false)) AS documents#90538, 10000002107 AS replenishmentRunId#90540L]
Input [3]: [openGoodsMovements#75051, plant#70763, product#70764]
(830) WriteFiles
Input [4]: [product#70764, location#90535, documents#90538, replenishmentRunId#90540L]
(831) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002107/0_1_10000002107, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002107/0_1_10000002107], Append, [product, location, documents, replenishmentRunId]
(832) ResultQueryStage
Output: []
Arguments: 1
(833) Filter
Input [3]: [openGoodsMovements#75051, plant#70763, product#70764]
Condition : (size(transform(openGoodsMovements#75051, lambdafunction(struct(documentType, PO, documentId, lambda x_28#90539.documentId, documentUuid, lambda x_28#90539.documentUuid, openQty, lambda x_28#90539.openQty, qtyUomUuid, lambda x_28#90539.qtyUomUuid, qtyUom, lambda x_28#90539.qtyUom, plannedDeliveryDatetime, lambda x_28#90539.availabilityDatetime), lambda x_28#90539, false)), false) > 0)
(834) Project
Output [4]: [product#70764, plant#70763 AS location#90535, transform(openGoodsMovements#75051, lambdafunction(struct(documentType, PO, documentId, lambda x_28#90539.documentId, documentUuid, lambda x_28#90539.documentUuid, openQty, lambda x_28#90539.openQty, qtyUomUuid, lambda x_28#90539.qtyUomUuid, qtyUom, lambda x_28#90539.qtyUom, plannedDeliveryDatetime, lambda x_28#90539.availabilityDatetime), lambda x_28#90539, false)) AS documents#90538, 10000002107 AS replenishmentRunId#90540L]
Input [3]: [openGoodsMovements#75051, plant#70763, product#70764]
(835) WriteFiles
Input [4]: [product#70764, location#90535, documents#90538, replenishmentRunId#90540L]
(836) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002107/0_1_10000002107, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002107/0_1_10000002107], Append, [product, location, documents, replenishmentRunId]
(837) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true